FR2902216A1 - Motional field generation module for group of pictures, has estimation unit for estimating motional field between two images, where one of images is separated from other image at distance higher than or equal to another distance - Google Patents

Abstract

The module has an estimation unit (110) i.e. estimator, for estimating a motional field between two images of a sequence of images, where one of the images is separated from the other image at a temporal distance higher than or equal to another temporal distance. A deducing unit (120) deduces another motional field between the latter image and a third image from the motional field estimated by the estimation unit. An independent claim is also included for a method for generating a motional field between images of a sequence of images.

Description

METHOD AND DEVICE FOR GENERATING MOTION FIELDS AND DEVICE

  Field of the Invention The invention relates to a device and a method for generating motion fields between a first image and a second image of a sequence of images. The invention also relates to a coding device for a sequence of images implementing said method of generating motion fields.

  2. State of the art Most image sequence encoding methods, such as MPEG or such as methods based on a t + 2D wavelet pattern (ie using a time analysis step t and a step of 2D spatial analysis), comprise a first temporal analysis step which makes it possible to reduce the temporal redundancy between successive images. Before coding, the image sequence is generally divided into groups of images (GOP - Group Of Pictures or GOF - Group Of Frames). With reference to FIG. 1, step 10 of the coding method consists of a temporal analysis of the images of the sequence. For this purpose, the images are generally motion compensated from motion vectors MV, for example by motion-compensated temporal filtering ("MCTF") in order to obtain different time frequency bands or else by prediction as is the case in encoding schemes based on the MPEG2 standard defined in ISO / IEC 13818-2 (entitled Information technology - Generic coding of moving pictures and associated audio information: Video). An image is motion compensated by using motion vectors MV previously generated by a motion estimation step 11. This step 11 may, for example, implement a conventional block matching estimation or a pel-recursive estimate. The resulting images of this time analysis are then coded in a step 12. During a multiplexing step 13, the motion data previously coded during a step 14 are combined with the coded data relating to the space coefficients. in order to create a single coded data stream. FIG. 2 illustrates a temporal analysis method according to the state of the art of a GOF of 4 images referenced A1, B1, C1 and D1 by Haar type filtering without an update step. . Such a method makes it possible to generate from the 4 images of the GOF a low resolution image L and 3 high resolution images H1, H2 and H3. In this case, the low resolution image is A1. The image H1 is generated from the image B1 and from the image A1 previously compensated in motion (MC) with respect to B1 with the aid of a first estimated motion field (ME) between Al and B1. More precisely, H1 = B1 (B) (A1) where MCAlB1 (A1) corresponds to the motion compensation of the image A1 from the estimated motion field from B1 to A1. The image H2 is generated from the image D1 and the image C1 previously compensated in motion (MC) with respect to D1 with the aid of a second estimated motion field (ME) between C1 and D1. More precisely, H2 = Dl ™ D1 (C1) where MCciDI (C1) corresponds to the motion compensation of the image C1 from the estimated motion field from D1 to C1. Finally, the image H3 is generated from the image C1 and the image A1 previously compensated in motion (MC) with respect to ci using a third estimated motion field (ME) between Al and D1. More precisely, H3 - Cl ûM ci (Al) where MCAlci (Al) corresponds to the motion compensation of the image Al from the estimated motion field from C1 to A1. Such a coding method requires generating for each GOF three fields of motion between images separated by a varying time distance. Indeed, the temporal distance between the images A1 and C1 is equal to twice the time distance between Al and B1. SUMMARY OF THE INVENTION The object of the invention is to overcome at least one of the drawbacks of the prior art. To this end, the invention relates to a device for generating a motion field between a first image and a second image of a sequence of images, the first and second images being spaced apart by a first temporal distance. The device comprises: - first means for estimating a motion field between the first image and a third image preceding the second image, the third image being spaced from the first image of a second temporal distance greater than or equal to the first temporal distance ; and second means for deriving a motion field between the first image and the second image from the motion field estimated by the first means. Preferably, the first means are adapted to estimate a motion field between two images spaced by a predetermined and fixed temporal distance equal to the second temporal distance. Advantageously, the first means are a motion estimator adapted to estimate a motion field between two images spaced by a predetermined and fixed temporal distance equal to the second temporal distance. According to one particular characteristic, this motion estimator operates sequentially on the images of the sequence of the images. This motion estimator may advantageously be an estimator using the temporal propagation of a motion field to estimate a current motion field from motion fields previously estimated.

  The invention also relates to a coding device for a sequence of images that comprises a motion estimation device according to the invention. The invention further relates to a method of generating the motion field between a first image and a second image of a sequence of images, the second image preceding the first image, the first and second images being spaced a first distance apart. time. The method comprises the steps of: estimating a motion field between the first image and a third image preceding the second image, the third image being spaced from the first image of a second temporal distance; and - deducing a motion field between the first image and the second image. According to a particular characteristic, the generated motion field is used by a coding method to encode the image sequence. Preferably, the coding method comprises a motion-compensated temporal filtering step. 4. Lists of Figures The invention will be better understood and illustrated by means of examples of advantageous embodiments and implementations, in no way limiting, with reference to the appended figures in which: FIG. 1 illustrates a coding method according to FIG. 'state of the art; FIG. 2 illustrates a time-based wavelet analysis method of a GOF structured image sequence of 4 images according to the state of the art; FIG. 3 illustrates the successive states of the memory in which are stored the images from which its motion field estimates; FIG. 4 represents a vector V1 of an estimated motion field and a vector V2 of a motion field deduced from the motion field to which V1 belongs; FIG. 5 illustrates a time-based wavelet analysis method of a GOF-structured image sequence of 4 images according to the invention; FIG. 6 illustrates a motion estimation module according to the invention; and FIG. 7 illustrates a device for coding an image sequence according to the invention. 5. Detailed Description of the Invention The invention relates to a method of generating motion fields between two images, not necessarily consecutive, of a sequence of images ordered according to a predefined order called a display order.

  The images of the sequence from which a motion field is generated are stored in their order of display in a memory of length N frames. With reference to FIG. 3 and for N = 3, the images DO, Al and B1 are stored in memory (state 1). A first MVDO <_B1 motion field is estimated between the first stored image DO and the last stored image B1. From this first estimated motion field, it is possible to deduce a second motion field MVA1 <_B1 by scaling the motion vectors of the first field MVpo <_B1. FIG. 4 represents a motion vector V1 of the first movement field MVpo <_B1 assigned to a block of pixels BIkB of the image B1 from which a vector V2 also assigned to this same block of pixels is deduced. For this purpose, the vector V2 belonging to the second motion field MVA1 <_B1 is deduced from the first motion vector V1 by scaling its coordinates V1 x and V1 y so that V2x = T1 * V1 x and V2y = T1 * V1 y, T2 T2

  where Ti is the time distance between the images A1 and B1 and T2 is the temporal distance between the images DO and B1. Then, the images stored in memory are shifted so that the image A1 occupies the place previously occupied by the image DO, the image B1 occupies the place previously occupied by the image A1 and the image C1 occupies the place previously occupied by the image B1, the image DO being no longer stored in memory (State 2). A third motion field MVA1 <_c1 is estimated between the first stored image A1 and the last stored image C1. From this third motion field MVA1 <_c1, it is possible to deduce a fourth motion field MVB1 <_c1 by scaling the motion vectors of the first field MVA1 <_c1. The images stored in memory are again shifted so that the image B1 occupies the place previously occupied by the image A1, the image C1 occupies the place previously occupied by the image B1 and the image D1 occupies the place previously occupied by the image C1, the image Al is then no longer stored in memory (State 3). A fifth motion field MVB1 <_D1 is then estimated between the first stored image B1 and the last stored image D1.

  From this fifth motion field MVB1 <_D1, it is possible to deduce a sixth motion field MVc1 <_D1 by scaling the motion vectors of the first field MVB1 <_D1. The method described for four consecutive images of the image sequence can be applied to the entire sequence. The estimated motion cambps may be by a block-pairing method, the estimated vectors being then assigned to blocks of pixels in an image as illustrated in Figure 4 or by a pel-recursive method, the estimated vectors are then assigned to pixels in an image. According to a particular characteristic of the invention, the estimation of a motion field is always carried out between two images spaced by a fixed predetermined time distance in this case equal to T2. Advantageously, the invention thus makes it possible to generate fields of motion between images spaced by a temporal distance that can vary (for example with reference to FIGS. 3 and 4), motion fields are estimated sequentially between two images spaced apart from one another. a distance T2 from these estimated motion fields the method according to the invention generates motion fields between images spaced by a temporal distance Ti different from T2) using a motion field estimation method operating sequentially and between images spaced by a fixed time distance. Such estimators are, for example, estimators using time propagation, i.e. using previously estimated motion fields to initialize new motion fields to estimate. According to an advantageous effect of the invention the deduced motion vectors are accurate insofar as each of the deduced vectors is assigned to the same pixel or block as the estimated vector from which it is deduced. Thus with reference to FIG. 4, the motion vector V2 deduced from the estimated motion vector V1 is assigned to the block blkB as is the vector V1.

  Preferably, the method of generating motion fields according to the invention is used by a method of coding an image sequence. The coding method according to the invention comprises a step of time analysis of the sequence. According to a particular embodiment, this time analysis step is performed by Haar-type filtering without an update step. Preferably, the sequence is divided into groups of images or GOFs, each group comprising a predetermined number (eg 4) of consecutive images. With reference to FIG. 5, in the particular case of groups of 4 images, the temporal analysis generates from 4 images A1, B1, C1, D1 of the initial sequence 3 high frequency images H1, H2, H3 and an image low frequency L. In the particular case of a time analysis step not involving an updating step, the low frequency image L is the image A1. The coding method according to the invention implements the motion estimation method according to the invention. Thus, the first motion field MVpo <_B1 is estimated between the image Do and the image Bi. The second motion field MVA1 <_B1 is then deduced from the first estimated field MVpo <_B1 as previously described. The image Al is then compensated in

  movement with respect to B1 from the second motion field MVA1 <_B1. The coding method according to the invention generates a high frequency image H1 from the image B1 and the image A1 compensated in motion with respect to the image B1 in the following manner: B1 ù MCA1BI (A1) H1 = or MCAlB1 (Al) corresponds to the motion compensation of the image A1 from the field MVA1 <_B1.

  The motion estimator then estimates the third motion field MVA1 <_c1 between the image A1 and the image C1. The image A1 is then compensated in motion with respect to ci using the estimated field MVA1 <_c1. The coding method according to the invention generates a high frequency image H3 from the image C1 and the image A1 compensated in motion with respect to the image C1 as follows: H3 - Cl ùM ci (Al) where MCAlci (Al) corresponds to the motion compensation of the image Al from the field MVA1 <_c1. The fourth motion field MVB1 <_c1 generated from this third motion field MVA1 <_c1 is not used for time analysis. The fifth motion field MVB1 <_D1 is estimated between B1 and D1 from which is deduced the sixth motion field MVC1 <_D1 between C1 and D1.

  The image C1 is then compensated in motion with respect to D1 from the sixth motion field MVC1 <_D1. The coding method according to the invention generates a high frequency image H2 from the image D1 and from the image C1 compensated in motion with respect to the image D1 in the following manner: H2 = D1 ù MCciDI (C1) where MCciDI (C1) corresponds to the motion compensation of the image C1 from the field MVC1 <_D1.

  The images stored in memory are then shifted so that the image A2 occupies the place previously occupied by the image D1, the image D1 occupies the place previously occupied by the image C1 and the image C1 occupies the place previously occupied by the image B1, the image B1 is then no longer stored in memory (State 4 in Figure 3). However, it is not necessary to estimate the motion field between the first image C1 stored in memory and the last image stored in memory A2. Indeed, in the particular embodiment described, the motion field MVC1 <_A2 is not used by the coding method according to the invention. With reference to FIG. 6, the invention also relates to a motion field generation module 1 which implements the method of generating motion fields according to the invention. The module 1 notably comprises a memory 100 making it possible to store N images of the sequence in the display order. For example in the previous embodiment N = 3. According to a particular embodiment of the invention, a motion estimation is performed between the first and last images stored in memory 110 in order to generate the MVDO motion fields <B1, MVA1 <-C1, MVB1 <-D1, and possibly MVC1 <-A2. This estimate is

  performed by a unit 110 for estimating a motion field. The unit 110 implements a conventional motion estimation method, for example by block matching or pel-recursive. The module 1 further comprises a unit 120 adapted to deduce the other motion fields MVA1 <-B1, MVB1 <-C1, MVC1 <-D1, MVD1 <-A2 from the motion fields

  estimated by the estimation unit 110. According to a variant, the module 1 also comprises a unit 130 making it possible to multiplex the motion fields generated by the estimation unit 110 and by the unit 120. According to an advantageous characteristic, the unit 1 unit 110 operates sequentially and between images spaced by a predetermined fixed time distance.

  Preferably, the unit 110 implements a pel-recursive motion estimation method using a temporal motion propagation, i.e. said method uses a previously estimated motion field to initialize the current motion field.

  Advantageously, the motion estimation module 1 is used by a coding device 2 of a sequence of images illustrated in FIG. 7. The coding device 2 comprises a motion estimation module 1 making it possible to generate fields movement according to the method of the invention. It further comprises a time analysis module 20 using the motion fields generated by the motion estimation module 1. Moreover, it comprises a coding module 21 for generating a coded data stream from the data generated. by the temporal analysis module 20. According to a particular embodiment, the coding module 21 comprises a spatial analysis unit implementing a spatial transform (eg a discrete cosine transform) and a coding unit. entropic. According to a particular characteristic, the coding device 2 also comprises a multiplexing module 22 making it possible to combine the data generated by the coding module 21 and the motion data generated by a coding unit of the motion vectors 23 in order to generate a unique encoded data stream. In Figures 6 and 7, the modules shown are functional units, which may or may not correspond to physically distinguishable units. For example, these modules or some of them may be grouped into a single component, or be functionalities of the same software. On the other hand, some modules may be composed of separate physical entities.

  Of course, the invention is not limited to the embodiments mentioned above. In particular, those skilled in the art can make any variant in the exposed embodiments and combine them to benefit from their various advantages. In particular the invention described for groups of 4 images can be applied to groups of more than 4 images or less than 4 images. The invention described for use by an encoding method comprising a wavelet based time analysis step may advantageously be applied to any coding method comprising a time analysis step requiring the generation of motion fields between images spaced apart from each other. a variable distance (for example between images spaced a distance Ti and between other images spaced a distance T2). Thus, the invention may advantageously be used by an encoding method based on the H.264 standard when the method uses the possibility offered by the standard to use several reference frames (multiple reference frames). In this case, according to the invention, an estimation of the motion field between the current image of index k and the preceding image of index k-N is performed. The intermediate motion fields between the current image and the images of index k-I with I <N are deduced from the estimated motion field between the current image of index k and the previous image of index k-N.20.

Claims (7)

claims   A device for generating a motion field (1) between a first image (BI) and a second image (A1) of a sequence of images, said first and second images being spaced apart by a first temporal distance ( T1), characterized in that it comprises: first means (110) for estimating a motion field between said first image (BI) and a third image (DO) preceding said second image (A1), said third image ( DO) being spaced apart from said first image (BI) by a second time distance (T2) greater than or equal to said first time distance (Ti); and second means (120) for deriving a motion field between said first image (BI) and said second image (A1) from the motion field estimated by said first means.   2. Device according to claim 1, characterized in that said first means (110) are a motion estimator adapted to estimate a motion field between two images of said sequence spaced by a predetermined and fixed temporal distance equal to said second distance temporal (T2).   3. Device according to claim 2, characterized in that said motion estimator is adapted to operate sequentially on the images of said sequence.   4. Encoding device (2) of a sequence of images characterized in that it comprises a device for generating a motion field (1) according to one of claims 1 to 3. 30   5. A method of generating a motion field between a first image (BI) and a second image (A1) of a sequence of images, said second image (A1) preceding said first image (BI), said first and second images (A1) second images being spaced apart by a first temporal distance (T1), characterized in that it comprises the following steps: estimating a motion field between said first image (BI) and a third image (DO) preceding said second image (A1), said third image (DO) being spaced apart from said first image (BI) by a second time distance (T2) greater than or equal to said first time distance (T1); and - deducing a motion field between said first image (B1) and said second image (Al).   The method of claim 5, characterized in that said generated motion field is used by a coding method to encode said sequence of images.   7. Method according to claim 6, characterized in that said coding method comprises a motion-compensated temporal filtering step. 15