KR20050023122A - System and method for segmenting - Google Patents

Abstract

Segmentation system 100 for segmenting a first image feature 214 of a first video image from an adjacent second image feature 216 of the first video image based on image characteristics such as color or luminance and motion. This is disclosed. Segmentation system 100 is based on block-based motion estimator 102 for estimating motion vectors 218-230 for blocks of pixels and based on motion vectors 218-230 of respective blocks of pixels. A motion segmentation unit 104 for segmenting a first video image into a first group 204 of connected blocks of pixels and a second group of connected blocks of pixels 206, wherein each of said image characteristics Based on the values, the first image feature from the second image feature 216 by pixel-based segmentation of a portion of the blocks of pixels of the first and second group 214, 216 of connected blocks of pixels. Pixel-based segmentation unit 106 for segmenting 214.

Description

System and method for segmenting

The present invention relates to a method of segmenting a first image feature of a first video image from an adjacent second image feature of a first video image.

The invention also relates to a segmentation system for segmenting a first image feature of a first video image from an adjacent second image feature.

The invention also relates to a motion estimator of pixel resolution for estimating a motion vector field comprising said segmentation system.

In addition, the present invention,

Receiving means for receiving a signal representing a series of video images,

A motion estimator of the pixel resolution for estimating a motion vector field from video images,

A motion compensated image processing unit for determining processed images based on the video images and the motion vector field.

Examples of methods of the kind described at the outset are known from US Pat. No. 6,075,875. The method includes obtaining a motion representation of corresponding pixels in a selected video image and a preceding video image for forming motion-segmented video image features, such as, for example, motion vectors. In addition, video image features are segmented according to their spatial image characteristics to form spatially-segmented video image features. As a result, the video image features are jointly segmented as a weighted combination of motion segmented video image features and spatially-segmented video image features. A disadvantage of the method according to the prior art is that it is a relatively complex method of motion-segmented video image features and spatially-segmented video image features. Note that two different quantities are weighted: motion based quantity and spatial image characteristic. The weighting of other quantities is not common.

1 shows schematically an embodiment of a segmentation system according to the invention;

2A schematically illustrates an initial configuration of blocks for block based motion segmentation.

FIG. 2B schematically illustrates the updated configuration of the block of FIG. 2A after one modification.

FIG. 2C illustrates the resulting configuration of blocks of block based motion segmentation. FIG.

3A schematically illustrates input blocks for pixel based segmentation.

3B schematically illustrates the initial edge of a horizontal block for pixel based segmentation.

3C schematically illustrates detected edges for the horizontal block pair of FIG. 3B.

FIG. 3D schematically illustrates the initial edge for a vertical block pair for pixel based segmentation. FIG.

3E is a schematic illustration of the last detected edge.

4A illustrates the inputs and outputs of a motion estimator to estimate a pixel accurate motion vector field.

4B illustrates the inputs and outputs of an alternative motion estimator to estimate a pixel accurate motion vector field.

5 schematically shows the components of an image processing apparatus according to the present invention;

It is an object of the present invention to provide a method of the kind of pixel accurate and relatively easy described at the outset.

An object of the present invention is a method for segmenting a first image feature 214 of a first video image from an adjacent second image feature of the first video image, wherein the first image feature is substantially in a first range. Having a plurality of pixels having respective values of an image characteristic that are values, and having motion associated with the second image feature between the first and second video images, the second image feature being substantially A method is achieved by the method, having a plurality of pixels having respective values of the image characteristic that are values of a second range different from values of one range,

Dividing the first video image into blocks of pixels,

Estimating motion vectors for respective blocks of pixels;

Segmenting the first video image into a first group of connected blocks of pixels and a second group of connected blocks of pixels by classifying a block of pixels based on the motion vectors of the respective blocks of pixels. Steps,

A pixel of the first and second group of connected blocks of pixels, determined to be located at a boundary between the first and second group of connected blocks of pixels, based on the respective values of the image characteristic Segmenting the first image feature from the second image feature by pixel-based segmentation of a portion of the blocks of data.

An advantage of the method according to the invention is that two segmentation operations are applied at the appropriate position. The result of block-based motion segmentation is used as input to pixel-based segmentation based on image characteristics. Pixel-based segmentation is applied to recover image feature boundaries of a small subset of the video image, such as blocks of object boundaries, ie, a boundary between the first and second groups of connected blocks of pixels. This is an efficient strategy that results in efficient computational resource usage because the detailed pixel-based segmentation process applies only to blocks at boundaries, not all blocks. Block based motion segmentation and pixel based segmentation are applied sequentially. That is, no complex weighting elements need to be coordinated between the results of the segmentation approach as in the prior art.

In an embodiment of the method according to the invention, segmenting the first video image into a first group of connected blocks of pixels and a second group of connected blocks of pixels is based on a motion model. An advantage of this embodiment is that it is a robust method. Most or all of the motion vectors of the test groups of connected blocks for the first group of connected blocks of pixels are applied to estimate the proper configuration of blocks to which the first image feature belongs. In other words, it is in the form of region fitting. Many segmentation methods consider blocks at the boundary of the segment to determine if the blocks correspond to a particular segment. The motion model can be based on rotation and / or transformation. Preferably, however, segmentation is based on an affine motion model.

In an embodiment of the method according to the invention, segmenting a first video image into the first group of connected blocks of pixels and to the second group of connected blocks of pixels,

Creating a first initial group of connected blocks of pixels for the first group of connected blocks of pixels, wherein the first initial group of connected blocks of pixels comprises a particular block of pixels; Steps,

Determining a first motion model for the first initial group of connected blocks of pixels;

A motion vector corresponding to the particular block of pixels estimated during the estimation of motion vectors for the respective blocks of pixels of the first video image and corresponding to the particular block of pixels based on the first motion model Calculating a first match error between the motion vectors,

Excluding said specific block of pixels, calculating a second motion model for a test group of connected blocks of pixels based on said first initial group of connected blocks of pixels,

The motion vector corresponding to the particular block of pixels estimated during the estimation of the motion vectors for the respective blocks of pixels of the first video image corresponding to the particular block of pixels based on a second motion model Calculating a second matching error between the motion vectors;

Determining whether said particular block of pixels corresponds to said first group of connected blocks of pixels based on said first and second matching error of pixels.

An advantage of this embodiment according to the invention is that it allows a scanning approach. It is tested repeatedly in block by block, ie from the top left to the bottom of the guess. The test means that for each block, the other blocks are evaluated where to merge to form a group of blocks corresponding to one of the image features. The advantage of the scanning approach is that it is a relatively easy to implement method.

In an embodiment of the method according to the invention, pixel-based segmentation is based on a spatial color model. In other words, the image characteristic upon which pixel-based segmentation is based is color. Color is a relatively good clue for segmentation. In this embodiment, two segmentation approaches are applied for proper scale resolution. Color is applied for high frequencies and low frequencies are applied for motion segmentation. Since chrominances are caused by texture, motion differences caused by different velocities of objects in the scene can be captured.

In an embodiment of the method according to the invention, the pixel based segmentation is based on a spatial luminance model. In other words, the image characteristic upon which pixel-based segmentation is based is luminance. Luminance is a relatively good clue for segmentation.

In an embodiment of the method according to the invention, said block of pixels of said first group of connected blocks of pixels 204c is determined to be located at a boundary between said first and second group of connected blocks of pixels. In a first block 302 of the portion of the step of the values of the image characteristic is detected. The step at the former corresponds to the edge of the first image feature. Preferably, the step is

Calculating a first average value of the image characteristic for the pixels of the first block;

Calculating a first difference measure based on the first average value and respective values of the pixels of the first block;

Calculating a second average value of the image characteristic for the pixels of the second block of pixels, wherein the second block of pixels corresponds to the second group of pixels and is connected to the first block;

Calculating a second difference measure based on the second average value and respective values of the pixels of the second block;

Generating a first test group of pixels excluding a particular pixel based on the first block and generating a second group of pixels comprising the specific pixel based on the second block,

Calculating a third mean value of the image characteristic for the pixels of the first test group;

Calculating a third difference measure based on the third mean value and the respective values of the pixels of the first test group;

Calculating a fourth average value of the image characteristic for the pixels of a second test group;

Calculating a fourth difference measure based on the fourth average value and the respective values of the pixels of the second test group;

Based on the first, second, third and fourth difference measurements, it is detected by determining whether the particular pixel belongs to a first image feature or the second image feature. An advantage of this embodiment is that it is a robust method. Most or all of the motion vectors of the test groups of connected blocks for the first group of connected blocks of pixels are applied to estimate the proper configuration of blocks to which the first image feature belongs. This is called a region fitting approach.

It is a further object of the present invention to provide a segmentation system of the kind described at the outset, which is arranged and relatively easy to provide pixel accurate segmentation.

The object of the present invention is a segmentation system for segmenting a first image feature 214 of a first video image from an adjacent second image feature of the first video image, the first image feature being substantially first Have a plurality of pixels having respective values of an image characteristic that are values of a range, and have a motion associated with the second image feature between the first video image and the second video image, the second image feature being substantially Achieved by the segmentation system, the segmentation system having a plurality of pixels with respective values of the image characteristic being values of a second range different from the values of the first range, wherein the segmentation system comprises:

Means for dividing the first video image into blocks of pixels,

A block-based motion estimator for estimating motion vectors for respective blocks of pixels,

Segmenting the first video image into a first group of connected blocks of pixels and a second group of connected blocks of pixels by classifying a block of pixels based on the motion vectors of the respective blocks of pixels. A motion segmentation unit for

 A pixel of the first and second group of connected blocks of pixels, determined to be located at a boundary between the first and second group of connected blocks of pixels, based on the respective values of the image characteristic A pixel-based segmentation unit for segmenting the first image feature from the second image feature by pixel-based segmentation of a portion of the blocks of blocks.

It is another object of the present invention to provide a motion estimator at pixel resolution to estimate a relatively easy pixel accurate motion vector field.

The object of the invention is achieved because a pixel accurate motion estimator comprises a segmentation system as claimed in claim 9. Alternatively, the pixel accurate motion estimator is provided with the results calculated by the segmentation system.

It is advantageous to apply an embodiment of a pixel accurate motion hearing aid according to the invention to an image processing apparatus as described earlier. The image processing apparatus may comprise additional components, for example, a display device for displaying the processed images or storage means for storing the processed images. The motion compensated image processing unit may support one or more of the following types of image processing.

De-interlacing: Interfacing is a common video broadcast procedure for alternating transmission of odd or even image lines. De-interlacing attempts to reconstruct the full vertical resolution, ie make odd or even lines available for each image simultaneously.

Up-conversion: A larger one's output images are calculated from a series of original input images. The output image is temporarily located between two raw input images.

-Temporary noise reduction. This may include spatial processing that results in spatial-temporal noise reduction.

Video compression, ie encoding or decoding according to eg the MPEG standard or the H26L standard.

Modifications and variations thereof may correspond to a segmentation system, modifications and modifications of a pixel accurate motion estimator, and modifications and modifications thereby to the image processing apparatus described above.

These and other aspects of the method, segmentation system, pixel accurate motion estimator of the image processing apparatus according to the present invention will be apparent from and elucidated with reference to the accompanying drawings, with reference to the implementations and embodiments described below.

Like reference numerals are used to refer to like parts throughout the figures.

1 schematically illustrates an embodiment of a segmentation system 100 according to the present invention. The segmentation system 100 is arranged to segment the first image feature 214 of the first video image from the adjacent second image feature 216 of the first video image. The first image feature 214 has a plurality of pixels, each having color values that substantially become values of the first range, and the second image feature 216 substantially differs from the values of the first range. It has a plurality of pixels each having color values that are values of. The first image feature 214 has a motion related to the second image feature 216 between the first video image and the second video image. The second video image may precede and follow the first video images. Provides a segmentation system 100 includes input connectors 108, pixel accurate segmentation results on receiving the signal representative of a video image, output connector 100 (S ^p) from. Segmentation system 100,

A block-based motion meter 102 for estimating motion vectors. This motion estimator 102 is arranged to estimate the motion vectors 218-230 for each of the blocks for which the first video image is divided by the splitting unit 103. This partitioning causes the pixels of the first video image to be clustered into blocks of pixels. Typically, a block of pixels contains 8 * 8 pixels. Preferably, motion estimator 102 is described by G. de Haan et al., "True-Motion Estimation with 3-D Recursive Search Block Matching", IEEE Transactions on circuit and systems for video technology, vol 3, no 5, 10 1993 Wall, as described on pages 368-379. Motion estimator 102,

The first video image by classifying blocks of pixels based on the motion vectors 218-230 of the respective blocks of pixels by connecting the first group 204c (see FIG. 2C) of the connected blocks of pixels and the connection of the pixels. A motion segmentation unit 104 arranged to segment into a second group of blocks 206c (see FIG. 2C),

Blocks of pixels of the first and second groups 204, 206 of connected blocks of pixels, determined to be located at a boundary between the first and second groups 204, 206 of connected blocks of pixels ( Provide a block-based motion vector field M ^B to the pixel-based segmentation unit 106 arranged to segment the first image feature 215 from the second image feature 216 by pixel-based segmentation of the portion of 302-306. do.

The operation of the motion segmentation unit 104 will be described in more detail with respect to FIGS. 2A-2C, and the operation of the pixel-based segmentation unit will be described in more detail with respect to FIGS. 3A-3E.

The block-based motion estimator 102, the motion segmentation unit 104 of the segmentation system 100, and the pixel-based segmentation unit 106 may be implemented using one processor. Usually, these functions are performed under the control of a software program product. During execution, a software program product is usually loaded into memory, such as RAM, and executed there. The program may be loaded from background memory, such as a ROM, hard disk, magnetic and / or optical storage medium. Alternatively, application specific integrated circuits provide the functionality disclosed.

In FIG. 2A, motion vector field 200 includes a number of vectors 218-230 for each block of pixels. These blocks of pixels are clustered into initial groups 202a-212a of the connected blocks of pixels. Or in other words, FIG. 2A schematically illustrates an initial configuration of blocks for block based motion segmentation. The number of initial groups of connected blocks corresponds to the image features, i.e. the maximum number of objects desired to segment in the video image. In these exemplary cases, each initial group 202a-212a of the connected groups of pixels includes 5 * 5 blocks. All blocks of pixels of a particular initial group of connected blocks have the same classification label. By the scanning approach, all blocks will be evaluated. This means that appropriate clusters or groups 202c-206c (see FIG. 2C) of the connected blocks of pixels are determined. This group of connected blocks of pixels 202c-206c corresponds to one of the image features 214, 216 in the first video image. Finding the appropriate groups is accomplished with a number of iterations, where for the blocks it is tested where the groups of connected blocks 202c-206c of pixels belong. This will be a fixed number of iterations. Alternatively, the process of evaluating the various intermediate groups 202b-212b of the connected blocks is stopped if the modifications of the various intermediate groups 202b-212b of the connected blocks do not have any positive effect. The process of evaluation is limited to topological constraints. Only blocks at the boundary of the connected groups 202a-212a and 202b-212b of the blocks are tested. Note that the blocks are located at the boundary of the middle group of blocks 202b-212b after multiple iterations. Applying only to blocks of boundaries differs from the well-known k-means approach which does not have this limitation. Evaluation of a particular block of pixels 232 includes a number of steps. This is described for a particular block of pixels 232 during the first scan (see FIG. 2A):

Creating a first initial group 204a of connected blocks of pixels comprising a specific block of pixels 232;

Determining a first motion model for the first initial group 204a of connected blocks of pixels,

The first motion model and the motion vector corresponding to the particular block 232 of pixels to be estimated while estimating the motion vectors 218-230 for the respective blocks of pixels of the first video image. Calculating a first match error between the motion vectors corresponding to the particular block 232 of pixels based on the match error being preferably at the Euclidean distance between the motion vectors. Based on the above steps,

Exclude the particular block 232 of pixels and calculate a second motion model for the test group 204b of connected blocks of pixels based on the first initial group 204a of connected blocks of pixels. Step (see 204b of FIG. 2B),

Based on a second motion model and a motion vector corresponding to the particular block 232 of pixels estimated while estimating motion vectors 218-230 for the respective blocks of pixels of the first video image Calculating a second match error between the motion vectors corresponding to the particular block 232 of pixels;

Determining whether the particular block 232 of pixels corresponds to the first group 204c of connected blocks of pixels based on the first and second matching error of the pixels. This means that the particular block 232 of pixels is tested to better match the first initial group 204a of the connected blocks of pixels or to the neighboring group 202a of the connected block of pixels. do. In this case, the particular block 232 of pixels better matches the latter group 202a of the blocks, whereby the neighboring group 201a of the connected blocks of pixels with respect to the particular block 232 It extends to the intermediate group 202b of the resulting blocks. The first initial group of blocks 204a is adapted to the intermediate group 204b of blocks except for the particular block 202.

Alternatively, additional matching errors are considered and calculated to determine where a particular block of pixels 232 belongs to the initial groups of blocks 202a-212a. For example, the third matching error is calculated for the second initial group of connected blocks of pixels as shown in FIG. 2A and the second test group 202b of connected blocks of pixels as shown in FIG. 2B. A fourth matching error for is calculated.

The test to see if a particular block of pixels 232 better matches the connected first initial group 204a of pixels or the neighboring group 202a of the connected blocks of pixels is a test of the connected block of pixels. It may also be performed by evaluating the first initial group 204a of the neighboring group of pixels and the neighboring group 202a of connected blocks of pixels. This means that there is no adaptation of these groups by moving a particular block of pixels 232 from the first initial group 204a of the connected blocks of pixels to the neighboring group 202a of the connected blocks of pixels. . In that case, motion models are calculated for both groups of connected blocks.

2B schematically illustrates an updated configuration of blocks for block based motion segmentation after one modification. The modification may be performed directly after all relevant blocks of the motion vector field have been evaluated. Related blocks refer to blocks located at the boundary of a group of connected blocks of pixels.

It is shown that a particular block of pixels 232 has been moved from a first initial group of connected blocks of pixels to a neighboring group 202a of horizontally located blocks. However, vertical movement is also possible, such as, for example, the movement of another block of pixels from the first initial group of blocks 204a to the group of blocks 210a of the underlying pixels.

After multiple iterations, multiple final groups 202c-206c of blocks of pixels are established. These final groups 202c-206c of blocks of pixels are the output of the block based motion segmentation. Figure 2c schematically illustrates the resulting configuration of the output, i. E. Block-based motion segmentation. The output is provided to the pixel-based segmentation unit 106 to detect additional processing, ie, the actual boundaries of the first image feature 214 and the second image feature 216.

Preferably, block based motion segmentation results for one image pair, such as, for example, a first image and a second image, may be applied for initialization of block based motion segmentation of a subsequent image pair.

3A schematically illustrates input blocks 302-306 for pixel-based segmentation. The motion segmentation result indicates that one of the blocks belongs to the first image video feature 214 and is labeled as 1, and two of the blocks 204 and 306 belong to the second image video feature 216 and as 2 Indicates that it is labeled. Pairs of neighboring blocks are evaluated to find the actual edge of the first image feature 214.

3B shows the initial edge 308 for horizontal pairs 203 and 304 of neighboring blocks. In a manner similar to that described above for block-based motion segmentation, pixel-based segmentation is performed. For pixels, it is meant that the pixels are evaluated for which group of connected pixels they correspond. This is based on a spatial color model based on the following assumptions: a concatenated pair of neighboring blocks 302, 304, connected with color values that substantially become values in the first range. There is a first group of pixels and a second group of pixels having color values that are substantially values of the second range. There is also a change or step in the color value between the first group of connected pixels and the second group of connected pixels. This means that for each pixel of the pair of neighboring blocks 302 and 304, each pixel evaluates which of the two groups of pixels it is connected to. Evaluation is an iterative process. This can be a fixed number of iterations. Alternatively, the process of evaluating various intermediate groups of connected pixels is stopped if the modification of the intermediate groups of connected pixels has no positive effect. The evaluation for specific pixels is

Calculating a first average value of the image characteristic for the pixels of the first block 302;

Calculating a first difference measure based on the first average value and respective values of the pixels of the first block 302;

Calculating a second average value of the image characteristic for the pixels of the second block 304 of pixels, the second block 304 of pixels corresponding to the second group of pixels and the first block Connected to 302, and

Calculating a second difference measure based on the second average value and respective values of the pixels of the second block 304,

Generate a first test group of pixels excluding a particular pixel based on the second block 302 and generate a second group of pixels comprising the particular pixel based on the second block 304 Steps,

Calculating a third mean value of the image characteristic for the pixels of the first test group;

Calculating a third difference measure based on the third mean value and the respective values of the pixels of the first test group;

Calculating a fourth average value of the image characteristic for the pixels of a second test group;

Calculating a fourth difference measure based on the fourth average value and the respective values of the pixels of the second test group;

Determining, based on the first, second, third and fourth difference measurements, whether the particular pixel belongs to the first image feature 214 or the second image feature 216. do.

After the multiple iterations, the last first group of pixels includes pixels corresponding to the first image feature and the last second group of pixels includes pixels corresponding to the second image feature. FIG. 3C schematically illustrates the detected edge 308 for the horizontal block pair of FIG. 3.

After the evaluation of the horizontally located pairs 302, 304 of the neighboring blocks is completed, a similar evaluation is started for the vertically located pairs 302, 306 of the neighboring blocks. 3D shows the initial edge 301 for this vertically located pair 302, 306 of neighboring blocks. 3E schematically shows the edge 301 finally detected. Evaluation of horizontally located pairs 302 and 304 of neighboring blocks and vertically located pairs 302 and 306 of neighboring blocks may be performed simultaneously.

Although only color has been discussed, other image characteristics may be applied for pixel-based segmentation, such as, for example, induced characteristics such as luminance or a combination of color and luminance or differences in color / luminance values between neighboring pixels.

4A schematically illustrates the input and outputs of a motion estimator 400 for estimating a pixel accurate motion vector field. The motion vector field M ^B calculated by the block-based motion estimator 102 is provided to the input connector 404 as described in connection with FIG. 1. Optionally, motion models calculated by motion segmentation unit 104 may also be provided. Because of the block resolution of the motion vector fields M ^B , the motion vectors 208-230 are not accurate for all the pixels of the first video image. However, at another input connector 406 of the motion estimator 400, the pixel accurate segmentation result ^SP may also be provided by the segmentation system 100 as described in connection with FIG. 1. By combining the information provided, the pixel accurate motion vector field can be determined. In particular, the pixels of the blocks located at the boundaries 214, 216 of the image features may be wrong. Preferably, motion vectors may be assigned to respective pixels corresponding to motion vectors estimated for neighboring blocks. The location of the appropriate neighboring blocks and thus the motion vector can be determined by the segmentation result ^SP . Usually, for the portion of pixels of the blocks located at the boundary 214, 216 of the image feature, the same motion vector as the motion vector of the block corresponding to the first image feature 214 is assigned, and for other portions of the pixels. Another motion vector equal to the motion vector of the block corresponding to the second image feature 216 adjacent to the first image feature 216 is assigned.

4B schematically illustrates the inputs and outputs of an alternative motion estimator 401 for estimating a pixel accurate motion vector field. The video images are provided to an input connector 402 of the motion estimator 401, and to another input connector 406 of the motion estimator 401, a pixel provided by the segmentation system 100 as described in connection with FIG. 1. Accurate segmentation result ^SP is provided. The pixel accurate motion estimator 401 is each image feature 214, 216 as segmented by the segmentation system 100 by comparing the pixels of the group of pixels with the preceding or subsequent corresponding pixels of the video images. Are arranged to calculate motion vectors for groups of pixels corresponding to. Preferably, the comparison is based on matching errors corresponding to the sum of absolute pixel value differences.

5 shows components of an image processing apparatus 500 according to the present invention. The image processing apparatus 500,

A receiving unit 502 for receiving a signal representing video images displayed after some processing is performed, which may be a broadcast signal received via an antenna or cable, but a video cassette recorder (VCR) or a digital versatile (DVD) A signal from a storage device, such as a disk, wherein the signal is provided at an input connector 510;

A processing unit 504 comprising a segmentation system 100 and a motion estimator 401 as described in relation to FIGS. 1 and 4A, respectively.

A motion compensated image processing unit 506,

Optional and comprises a display unit 508 for displaying the processed images.

The motion compensated image processing unit 506 needs motion vectors and images as input. The motion compensated image processing unit 506 may support one or more forms of image processing such as de-interlacing, up-conversion, temporal jam reduction, and video compression.

It is to be noted that the above-described embodiments illustrate rather than limit the invention, and that those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of components or steps not listed in a claim. Singular expressions of components do not exclude the presence of a plurality of such components. The present invention may be implemented by hardware and some suitably programmed computer comprising several unique components. The unit claims several means of enumerating, some of which may be embodied by the same item and one of the hardware.

Claims (17)

A method for segmenting a first image feature 214 of a first video image from an adjacent second image feature 216 of the first video image, wherein the first image feature 214 is substantially in a first range. The second image feature 216 has a plurality of pixels having respective values of an image characteristic that are values and has a motion associated with the second image feature 216 between the first video image and the second video image; ) Has a plurality of pixels having respective values of the image characteristic that are substantially values of the second range different from the values of the first range, Dividing the first video image into blocks of pixels, Estimating motion vectors 218-230 for the respective blocks of pixels; Connect the first video image to the first group of connected blocks of pixels 204c and the pixels by classifying the blocks of pixels based on the motion vectors 218-230 of the respective blocks of pixels. Segmented into a second group of blocks 206c, and then located at a boundary between the first and second groups 204c, 206c of connected blocks of pixels, based on the respective values of the image characteristic. From the second image feature 216c by pixel-based segmentation of the portion of the blocks of pixels of the first and second group 204c, 206c of the connected blocks of pixels, Segmenting 214c). 2. The method of claim 1, wherein segmenting the first video image into the first group 204c of connected blocks of pixels and the second group 206c of connected blocks of pixels is based on a motion model. , Way. 3. The method of claim 2, wherein segmenting the first video image into the first group 204c of connected blocks of pixels and the second group 206c of connected blocks of pixels comprises a combined motion model ( based on an affine motion model. 4. The method of claim 2 or 3, wherein segmenting the first video image into the first group 204c of connected blocks of pixels and the second group 206c of connected blocks of pixels, Creating a first initial group 204a of connected blocks of pixels for the first group 204c of connected blocks of pixels, wherein the first initial group 204a of connected blocks of pixels is The step comprising a specific block of pixels 232; Determining a first motion model for the first initial group 204a of connected blocks of pixels, The first motion model and the motion vector corresponding to the particular block 232 of pixels to be estimated while estimating the motion vectors 218-230 for the respective blocks of pixels of the first video image. Calculating a first match error between the motion vectors corresponding to the specific block 232 of pixels based on the following; Exclude the particular block 232 of pixels and calculate a second motion model for the test group 204b of connected blocks of pixels based on the first initial group 204a of connected blocks of pixels. Steps, A motion vector corresponding to the particular block 232 of pixels to be estimated while estimating motion vectors 218-230 for the respective blocks of pixels of the first video image to the second motion model; Calculating a second matching error between the motion vectors corresponding to the specific block 232 of pixels based on the results; Determining whether the particular block of pixels (232) corresponds to the first group (204c) of connected blocks of pixels based on the first and second matching error of pixels. The method of claim 1, wherein the pixel-based segmentation is based on a spatial color model. The method of claim 1, wherein the pixel-based segmentation is based on a coherent luminance model. 7. The method of claim 5 or 6, wherein the pixels of the first group 204c of connected blocks of pixels are determined to be located at the boundary between the first and second groups of connected blocks of pixels. In the first block 302 of the portion of the block, a step in the values of the image characteristic is detected The method of claim 7, wherein the step, Calculating a first average value of the image characteristic for the pixels of the first block 302; Calculating a first difference measure based on the first average value and the respective values of the pixels of the first block 302; Calculating a second average value of the image characteristic for the pixels of the second block of pixels 304, wherein the second block 304 of pixels corresponds to the second group of pixels and the first block Connected to 302, and Calculating a second difference measure based on the second average value and the respective values of the pixels of the second block 304; Generate a first test group of pixels excluding a particular pixel based on the first block 302, and generate a second group of pixels comprising the particular pixel based on the second block 304 Steps, Calculating a third mean value of the image characteristic for the pixels of the first test group; Calculating a third difference measure based on the third mean value and the respective values of the pixels of the first test group; Calculating a fourth average value of the image characteristic for the pixels of a second test group; Calculating a fourth difference measure based on the fourth average value and the respective values of the pixels of the second test group; Determining whether the particular pixel belongs to a first image feature 214 or the second image feature 216 based on the first, second, third and fourth difference measurements Detected method. A segmentation system for segmenting a first image feature 214 of a first video image from an adjacent second image feature 216 of the first video image, wherein the first image feature 214 is substantially in a first range. Having a plurality of pixels having respective values of an image characteristic that are values of and having a motion associated with the second image feature 216 between the first video image and the second video image, the second image feature ( 216 is a segmentation system having a plurality of pixels having respective values of the image characteristic that are substantially values of a second range different from the values of the first range, wherein Means (103) for dividing the first video image into blocks of pixels, A block-based motion estimator 102 for estimating motion vectors 218-230 for the respective blocks of pixels, Connect the first video image to the first group of connected blocks of pixels 204c and the pixels by classifying the blocks of pixels based on the motion vectors 218-230 of the respective blocks of pixels. A motion segmentation unit 104 for segmenting into the second group 206c of blocks,  The first and second connected blocks of pixels determined to be located at a boundary between the first and second groups 204c, 206c of connected blocks of pixels, based on the respective values of the image characteristic. Pixel-based segmentation unit 106 for segmenting the first image feature 214 from the second image feature 216 by pixel-based segmentation of a portion of blocks of pixels of a second group 204c, 206c Including, the segmentation system. 10. A motion estimator (400, 410) of pixel resolution for estimating a motion vector field, comprising the segmentation system (100) as claimed in claim 9. 11. The motion estimator 400 as claimed in claim 10, wherein the motion vectors estimated for each block based on the segmentation of the first image feature 214 from the second image feature 216. Assigning (218-230) to the respective pixels of the first video image. A motion estimator 401 as claimed in claim 10, wherein the pixels of the first image feature 214 are compared to the first image feature 214 by comparison with corresponding pixels of the second video image. A motion estimator 401 of pixel resolution, arranged to estimate a new motion vector for. As the image processing apparatus 500, Receiving means 502 for receiving a signal representing a series of video images, A motion estimator 504 of pixel resolution as claimed in claim 10 for estimating a motion vector field from the video images, A motion compensated image processing unit (506) for determining processed images based on the video images and the motion vector field. 14. The image processing apparatus (500) of claim 13, wherein said motion compensated image processing unit (506) is designed to perform video compression. 14. The image processing apparatus (500) of claim 13, wherein said motion compensated image processing unit (506) is designed to reduce noise in a series of images. 14. The image processing apparatus (500) of claim 13, wherein the motion compensated image processing unit (506) is designed to de-interlace a series of images. 14. The image processing apparatus (500) of claim 13, wherein said motion compensated image processing unit (506) is designed to perform up-conversion.