Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
  • H04N19/51—Motion estimation or motion compensation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/14—Picture signal circuitry for video frequency region
  • H04N5/144—Movement detection
  • H04N5/145—Movement estimation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion

Definitions

• the invention relates to a method and device for estimating the dominant movement in a video plane. More specifically, the method is based on the analysis of the motion fields transmitted with the video in compression schemes using motion compensation. Such schemes are implemented in the MPEG-1, MPEG-2 and MPEG-4 video compression standards. There are known motion analysis methods based on the estimation, from motion vectors originating from compressed video streams of MPEG type, of a motion model which is most often refined:
• the vectors of the compressed video stream which serve as the basis for the analysis of the movement do not always reflect the reality of the actual apparent movement of the image. Indeed, these vectors have been calculated in order to minimize the amount of information to be transmitted after motion compensation, not to estimate the physical motion of the pixels in the image.
• the robust regression is the method of the least median of the squares which consists in seeking, among a set of lines j, ri, j being the residue of the ith sample of coordinates xi, ui or yi, vi , compared to a straight line j, that providing the median value of the set of residual squares which is minimal:
• the search for the least median of the squares of the residues is applied to a predefined number of lines each determined by a couple of samples drawn randomly in the space of representation of the movement considered.
• the invention also relates to a device for implementing the method.
• the method allows the implementation of robust methods of identifying the movement model at a reduced cost.
• the main interest of the method described in the invention lies in the use of a space of judicious representation of the components of the motion vectors, which makes it possible to reduce the identification of the parameters of the motion model to a double regression linear.
• a flow diagram of the method of detecting the dominant movement The characterization of the dominant movement in a sequence of images requires the identification of a parametric model of apparent dominant movement. In the context of the exploitation of motion vector fields from compressed video streams, this model must represent the apparent motion in the 2D image plane. Such a model is obtained by approximating the projection onto the image plane of the movement of objects in three-dimensional space.
• the 6-parameter affine model (a, b, c, d, e, f) presented above is commonly adopted in the literature.
• the proposed method consists, basically, in identifying this parametric model of the movement, from fields of motion vectors which are provided in the video stream in order to effect the decoding thereof, when the coding principle calls upon techniques. motion compensation as used for example in the MPEG-1, MPEG-2 and MPEG-4 standards.
• the method described in the invention is also applicable to motion vector fields which would have been calculated by a separate process from the images constituting the processed video sequence.
• (x ⁇ Yg) 1 coordinates of the reference point for the approximation of the 3D scene filmed by the camera into a 2D scene; this reference point will be assimilated to the point of coordinates (0, 0) 'of the image
• t x , t y J vector representing the translation component of the movement
• k divergence term representing the zoom component of the movement
• ⁇ rotation angle of the movement around the camera axis.
• the process of estimating the parameters of the simplified model of movement is based on the application of a linear regression of robust type in each of the spaces of representation of the movement.
• Linear regression is a mathematical operation which determines the line which best fits a cloud of points, for example by minimizing the sum of the squares of the distances from each point to this line.
• This operation is, in the context of the invention, implemented using a robust statistical estimation technique, so as to guarantee a certain insensitivity to the presence of outliers in the data.
• the estimation of the dominant movement model must be freed: - from the presence in the image of several objects, some of which follow secondary movements distinct from the dominant movement,
• the motion vectors transmitted in a compressed video stream have been calculated with the aim of minimizing the amount of residual information to be transmitted after compensation for movement and not with the aim of providing the real movement of the objects constituting the imaged scene.
• Figure 8 summarizes the different stages of the method for estimating the dominant movement in the sequence. Each of these steps is described in more detail below.
• a first step 1 performs a normalization of the motion vector fields each associated with an image of the processed video sequence. These vector fields are assumed to have been calculated prior to the application of the algorithm, using a motion estimator.
• the estimation of the movement can be carried out for rectangular blocks of pixels of the image, as in the methods of pairing of blocks of pixels called "block-matching", or provide a dense vector field, where a vector is estimated. for each pixel in the image.
• the present invention preferentially, but not exclusively, deals with the case where the vector fields used have been calculated by a video encoder and transmitted in the compressed video stream for decoding purposes.
• FIG. 3 illustrates point clouds obtained after construction of these two spaces from a field of normalized motion vectors.
• FIGS 4 to 7 show some examples of possible configurations.
• the next step 3 performs a robust linear regression for each of the movement representation spaces, with the aim of separating the data points representative of the real dominant movement from those corresponding either to the movement of secondary objects in the image, or vectors which do not translate the physical movement of the pixels with which they are associated.
• robust estimation techniques There are several families of robust estimation techniques.
• the regression lines are calculated so as to satisfy the criterion of the least median of the squares.
• the calculation method briefly presented below, is described more fully in paragraph 3 of the article by P. Meer, D. Mintz and A. Rosenfeld “Robust Regression Methods for Computer Vision: A Review", published in International Journal of Computer Vision, volume 6 no.1, 1991, pages 59 to 70.
• n, j the residue of the i th sample of a space representing the movement in which we seek to estimate the set E j of the regression parameters (slope and intercept of the regression line), we calculate Ej so as to meet the following criteria: min (medii 2 :)
• the residue n, j corresponds to the residual error ⁇ u ⁇ or ⁇ v , - - depending on the representation space considered - associated with the modeling of the i th sample by the regression line of parameters E j .
• the solution to this non-linear minimization problem requires a search for the line defined by Ej among all possible lines. In order to restrict the calculations, the search is limited to a finite set of p regression lines, defined by p pairs of points drawn randomly from the samples of the representation space studied. For each of the p lines, the squares of the residuals are calculated and sorted so as to identify the square of the square residue which has the median value. The regression line is estimated as the one that provides the smallest of these median values of the squares of the residuals.
• the regression lines obtained by robust estimation in each representation space are then used to identify the outliers. For this purpose, we calculate, as a function of the median value of the square of the residue corresponding to the best regression line found, a robust estimate ⁇ of the standard deviation of the residues associated with the non-aberrant samples, under the assumption that they follow a Gaussian distribution, and we label as an aberrant sample any sample whose absolute value of the residue exceeds K times ⁇ . It is advantageous to fix the value of K at 2.5. Still in this step 3, we finally perform classic linear regressions, not robust, on the samples of each space of representation, excluding samples identified as outliers. These regressions provide refined estimates of the parameters (ao, bo) and (a ⁇ , b ⁇ ) which will be used in the rest of the process.
• the next step 4 performs a linearity test of the regression lines in each of the representation spaces.
• the purpose of this test is to verify that the point clouds in each space are effectively approximately distributed along lines, which does not guarantee the existence - systematic - of a regression line.
• the linearity test is carried out, in each representation space, by comparing to a predetermined threshold the standard deviation of the residues resulting from the linear regression relating to the non-outlier samples.
• the value of the threshold depends on the time normalization applied to the motion vectors in step 1 of the method. In the case where, after normalization, each vector represents a displacement corresponding to the time interval separating two interlaced frames, ie 40 ms for a transmission at 50 Hz, this threshold can advantageously be set at 6.
• step 5 consists in verifying that the slopes ao and ai, which provide a double estimate of the divergence parameter k of the motion model, do not differ significantly.
• the equality test of two regression slopes is a known problem, which is dealt with in some statistical works; we can for example consult the chapter devoted to analysis of variance in CR Rao's book “Linear Stafistical Inference and its Applications” published by Wiley editions (2 nd edition). This test is carried out in a conventional manner by calculating an overall regression slope relating to the set of non-outlier samples of the two representation spaces of the motion vector field.
• the value of the divergence coefficient k of the dominant motion model is estimated by the arithmetic mean of the regression slopes a 0 and ai obtained in each of the representation spaces.
• the parameters t x and t y are estimated respectively by the values of the intercepts bo and bi from the linear regressions in the representation spaces.
• the vector ⁇ (k, t x , t y of the estimated parameters is used to decide the category in which to classify the dominant movement, namely: - static,
• the classification algorithm is based on nullity tests of the model parameters, in accordance with the table below:
• the nullity tests of the estimates of the parameters of the models can be carried out by simple comparison with a threshold of their absolute value. More sophisticated techniques, based on statistical modeling of the distribution of data can also be used. In this statistical framework, an example of an algorithm for deciding the nullity of model parameters based on likelihood tests is presented in the article by P. Bouthemy, M. Gelgon and F. Ganansia entitled "A unified approach to shot change detection and camera motion characterization ”, published in the journal IEEE Circuits and Systems for Video Technology volume 9 no. 7, October 1999, pages 1030 to 1044.
• One application of the invention relates to video indexing from the selection of key images.
• the video indexing process generally begins with a pre-processing, which aims to restrict the volume of information to be processed in the video stream to a set of key images selected in the sequence.
• the video indexing treatments, and in particular the extraction of the visual attributes, are carried out exclusively on these key images, each of which is representative of the content of a segment of the video.
• all of the keyframes should form a comprehensive summary of the video, and redundancies between the visual content of the keyframes should be avoided, so as to minimize the computational burden of the indexing process.
• the method of estimating the dominant movement within each video plane makes it possible to optimize the selection of the key images, within each plane, with respect to these criteria, by adapting it to the dominant movement. .
• the described method can also be used for the generation of metadata.
• the dominant movements often coincide with the camera movements when shooting the video.
• Some directors use specific camera movement sequences to communicate certain emotions or sensations to the viewer.
• the method described in the invention can make it possible to detect these particular sequences in the video, and therefore to provide metadata relating to the atmosphere created by the director in certain passages of the video.
• Another application of dominant motion detection is the detection or aid in the detection of ruptured planes. In fact, an abrupt change in the properties of the dominant movement in a sequence can only be caused by a break in the plane.
• the method described in the invention allows the identification, in each image, of the support of the dominant movement. This support indeed coincides with all the pixels whose associated vector has not been identified as an outlier, in the sense of the dominant movement.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Image Analysis (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)

Priority Applications (6)

Applications Claiming Priority (2)

Publications (1)

Family

ID=8870690

Family Applications (1)

Country Status (9)

Cited By (2)

Families Citing this family (18)

Citations (1)

Family Cites Families (9)

• 2001
  • 2001-12-19 FR FR0116466A patent/FR2833797B1/fr not_active Expired - Fee Related
• 2002
  • 2002-12-12 WO PCT/FR2002/004316 patent/WO2003055228A1/fr not_active Ceased
  • 2002-12-12 EP EP02805377A patent/EP1468568A1/fr not_active Withdrawn
  • 2002-12-12 CN CNB028258886A patent/CN100411443C/zh not_active Expired - Fee Related
  • 2002-12-12 KR KR1020047009616A patent/KR100950617B1/ko not_active Expired - Fee Related
  • 2002-12-12 MX MXPA04005991A patent/MXPA04005991A/es active IP Right Grant
  • 2002-12-12 US US10/499,560 patent/US20050163218A1/en not_active Abandoned
  • 2002-12-12 AU AU2002364646A patent/AU2002364646A1/en not_active Abandoned
  • 2002-12-12 JP JP2003555818A patent/JP4880198B2/ja not_active Expired - Fee Related

Patent Citations (1)

Non-Patent Citations (7)

Also Published As

Similar Documents

Legal Events