FR2864300A1 - Person localizing process for e.g. video telephone scenes filtering field, involves choosing active contour algorithm for video image to apply to initial contour for obtaining final contour localizing person to be extracted in image - Google Patents

Abstract

The process involves performing an initial segmentation of a video image based on an average saturation of the image and its comparison to a preset threshold. A model is applied to the segmented image and an initial contour of the image is deducted. An active contour algorithm for the image is chosen and applied to the initial contour for obtaining a final contour localizing a person to be extracted in the image. An independent claim is also included for a computer program product having instructions for implementing a process of localizing a person in a video image.

Description

PROCESS FOR LOCALIZATION AND FUZZY SEGMENTATION OF A

PERSON IN A VIDEO IMAGE

  The present invention relates to a method of locating a person in a video image. It also relates to a fuzzy segmentation method of said person.

  The invention finds a particularly advantageous, but not limiting, application in the field of filtering videophone scenes. However, other areas of applications may be considered such as surveillance or television transmission by ADSL.

  In applications related to videophone services, intended for the general public in particular, there are two types of difficulties, namely: io in the residential field, the fact of showing an image of his personal environment is often felt by users as an intrusion in their private life. This violation of privacy is a significant psychological barrier to the use of home video calling.

  - in the field of mobile phones, another type of difficulty arises which is linked to the fact that transmitting a large number of complex information (streets, cars, buildings, moving background, ...) only results in to scramble the videophone message that the nomadic user wishes to send.

  In both situations, but for different reasons, it is necessary to eliminate the background of the image to keep only people in conversation. For residential videotelephony, the advantage is to preserve the privacy of users. For mobile video telephony, the advantage is to be able to increase the compression ratio by transmitting only the useful image elements.

  To achieve these results, it is technically necessary to be able to segment the video image to distinguish from the rest of the image that we want to eliminate the areas of interest that we want to keep that is, those in which the person you want to isolate is present. It is clear that this segmentation operation, said fuzzy, requires a prior operation of locating the person in the video image.

  Known localization methods include the use of a neural network to classify pixels (pixels) as part of the face or not. The disadvantage of this approach lies in the computation time which in general does not allow to do real-time processing.

  There are also other localization methods known as deformable models. These models are based on an active contour algorithm whose general principle consists firstly in isolating the subject to be extracted in an approximate initial contour and in changing the contour by algorithmic deformation to a final contour constituting a good approximation as to the location. of the subject of interest in the image.

  In some active contour algorithms, the evolution of the contour is guided by the simultaneous optimization of two criteria. The first measure the geometric regularity of the contour, relying for example on the calculation of its local curvature. The second one measures a property of the luminous intensity along the points of the image traversed by the deformable contour, for example the norm of the gradient of the luminous intensity in order to favor the attraction of the contour towards points of strong contrast. These methods are effective when the user can initialize the model roughly around the region of interest and therefore require an initialization close to the solution, which requires a priori knowledge of the position of the person to extract in the picture. As a result, no practical implementation of these algorithms can be envisaged. Finally, it will further be noted that remote initialization of the solution requires frequent re-parameterizations which may present difficulties in adapting to the geometry of the studied form.

  Also, the technical problem to be solved by the object of the present invention is to propose a method of locating a person in a color coded video image according to a given color coding system, said method using at least one contour algorithm asset for determining a final contour of the person to be extracted from an approximated initial contour, a method which would make it possible to obtain an automatic location in an optimal processing time according to the complexity levels of the video sequence, making possible a Real-time implementation, good temporal stability and easy use by reducing the number of parameters to be set by the user.

  The solution to the technical problem posed consists, according to the present invention, in that said method comprises the following steps: io -a) determination of an approximate initial contour of the person to be extracted, consisting of: -a1) transforming the system of color coding the image into a TSL encoding, -a2) performing an initial segmentation of the image from the calculation of the mean saturation Sm of the image and its comparison with two predetermined thresholds S1 and S2 (> S1 ): -a21) if Sm> S2: apply to the image a flesh tint filter, -a22) if S1 <Sm <S2: apply to the image a histogram equalization operation, then a tint filter flesh, -a23) if Sm <S1: apply to the image a histogram equalization operation followed by a histogram thresholding operation, then a first segmentation operation by the movement of the image, -a3 ) apply a shape model to the image thus segmented and deduce from it said initial initial outline oché, -b) choice of an active contour algorithm, -c) application of said active contour algorithm to the approximate initial contour to obtain the final contour locating the person to extract in the video image.

  Thus, since the first step a) of searching for an approximate initial contour is carried out systematically over the entire video image, it is understood that the localization method according to the invention does not require a knowledge of priori topology of the subject to detect and therefore lends itself to automatic image processing.

  Furthermore, the reference to the only average saturation parameter Sm in order to define the type of processing to be applied to achieve the initial segmentation of the image considerably simplifies the localization method, which is the subject of the invention, by limiting the parameterization and by associating a definite treatment according to the value of this parameter. This results in the possibility of real-time implementation of the method of the invention, which constitutes a substantial advantage over other known methods of localization.

  As will be seen in detail below, the localization method according to the invention can use two types of active edge algorithm: the algorithm known as B-snake described for example in the following publications: "Active Contour Models: Overview, Implementation and Applications "(S. Menet, P. Saint-Marc and G. Medioni, IEEE, 1990, pp. 194-199) and" Representation of Range Data with B-Spline Surface Patches "(Chia-Xei Liao end G.Medioni, IEEE, 1992, pp. 745-748).

  an algorithm called B-snake-GVF which is an algorithm derived from the previous one which takes into account the external force known as GVF (Gradient Vector Flow), described in the following publications: Snakes, Shapes, and Gradient Vector Flow "(Chenyang Xu, IEEE Transactions on Image Processing, Volume 7, No. 3, March 1998)," Gradient Vector Flow: A New External Force for Snakes "(Chenyang Xu and Jerry L. Prince, IEEE Proc. Rec. P. (CVPR'97), pp. 66-71) and "Global Optimality of Gradient Vector Flow" (Chenyang Xu and Jerry L. Prince, 2000 Conference on Information Sciences and Systems, Princeton University, March 15-17 , 2000).

  The B-snake algorithm is the one that is most conventionally used. Its implementation is simple and its calculation time relatively fast. On the other hand, it requires an initial contour quite close to the object to be extracted. Indeed, if the initial contour is too far, the convergence towards the subject is ill easy and especially very slow.

  The B-snake-GVF algorithm is much more robust and leads to good results even in the case of a subject to extract having concavities or if the initial contours are far. On the other hand, its implementation is more complex than for the B-snake algorithm, but its main disadvantage lies in the fact that it is not usable if the video image contains two subjects of interest too close together, it occurs then interference making this algorithm unusable.

  In this respect, it can be recalled that the principle of active contour algorithms is based on the constant minimization of the energy AND of the curve representing the contour. This energy is defined so as to present a local minimum on the outer contour of objects of an image. Thus the initial contour changes to its final form.

  In the B-snake-GVF algorithm, the expression of the total energy to be minimized is: Eext is and is experimental coefficients of adjustment and% (sn,) = exp 'rj * Sm) a coefficient which makes it possible to select between the segmentation modes by the conventional B-snake algorithm and the B-snake-GVF algorithm, depending on the value of the average saturation Sm of the image.

  The invention proposes an optimization in the choice of the algorithm to be used consisting in that step b) of choosing an active contour algorithm comprises the following steps: -b1) determining the number NBP of people in the video image, 25 -b2) choice of the algorithm: -b21) if NBP = 1: choice of the algorithm B-snake-GVF, -b22) if NBP> 1: choice of the algorithm B-snake.

  In this way, we obtain that if only one person is present in the image, the more robust B-snake-GVF algorithm is used for an efficient treatment of the contour even under difficult initial conditions. On the other hand, if several people are present, we use the B-snake algorithm which in this context gives the best results.

  It should also be pointed out that in order to refine the initial segmentation result obtained in the case of an average saturation Sm lying between the two thresholds S1 and S2, the invention provides that step a22) is completed by a second operation. of segmentation by the movement of the image. In a general manner, said second operation of segmentation by the movement of the image is a simple difference image method, as will be described later.

  The localization method according to the invention as just characterized makes it possible to define a fuzzy segmentation method of a person in a video image which, in the case of the use of a contour algorithm unique, is remarkable, according to the invention, in that said method consists in locating said person by means of the previous location method, then in performing the following fuzzy segmentation operations: -dl) compute a confidence index IC, - d2 ) compare said confidence index IC with a predetermined threshold ICs: - d21) if IC≥ICs: perform a simple segmentation of the image by multiplying the inside and the outside of the final contour by 1 and 0 respectively, -d22) if IC <ICs: perform a complex segmentation of the image by multiplying by 1 the inside of the final contour and by a number between 0 and 1 provided by a Gaussian filter the outside of said contour.

  In the case where the localization method, which is the subject of the invention, implements, as desired, one or the other of two active contour algorithms according to the number NBP of persons present in the video image, said method Segmentation is remarkable, according to the invention, in that said method consists of locating said person by means of the preceding localization method, then performing the following fuzzy segmentation operations: d1) calculating a confidence index CI, -d2 ) comparing said confidence index CI with a predetermined threshold ICs: - d21) if IC≥ICs and if IC <lCs with NBP = 1: performing a simple segmentation of the image by multiplying respectively by 1 and 0 the interior and the outside of the final contour, -d22) if IC <lCs with NBP> 1: perform a complex segmentation of the image by multiplying by 1 the inside of the final contour and by a number between 0 and 1 provided by a Gaussian filter outside said contour.

  The following description with reference to the accompanying drawings, given as non-limiting examples, will make it clear what the invention consists of and how it can be achieved.

  Fig. 1 is a diagram showing a fuzzy location and segmentation method of a person in a video image according to the invention.

  FIG. 2 is a diagram representing step A of initialization of the method of FIG.

  Fig. 3 is a diagram showing a method of determining the NBP number of persons in the video image under consideration.

  FIG. 4 is a diagram illustrating two examples of application of the fuzzy localization and segmentation method according to the invention.

  In Figure 1 is schematically shown a method of fuzzy segmentation of a person in a video image.

  This method essentially comprises four steps designated by the letters A, B, C, and D. Steps A and B correspond to the implementation of a method for locating the person that is to be extracted from the image. video considered. Step C is the fuzzy segmentation step itself which consists in detaching the person from the background or background of the image by treating the image so as to make it blurry or eliminate it, without affecting the rendering of the image. visual of the person. The last step D is a data processing intended in particular to determine the temporal evolution of the useful parameters in order to update the contour of the person frame by frame and to be able to apply the best location algorithm.

  In the context of the invention, the locating method described here is of the type using at least one active contour algorithm for determining a final contour of the person to be extracted from an approximate initial contour.

  The first step of the locating method according to the invention is represented by step A of FIG. 1 and consists in defining an initial contour approximated to the person to be extracted. This operation is performed by an automatic initialization module, the implementation of which will now be described with reference to FIG. 2. Previously, it is necessary to insist on the automatic character of this initialization step, in the sense that, unlike to the other known algorithms, the initial approach contour is, according to the invention, determined by a global analysis of the video image considered, without it being necessary to have a priori knowledge of the position of the person sought in this image.

  As indicated in FIG. 2, step A of the localization method according to the invention begins with the transformation of the color coding of the video image (generally the standard RGB (Red-Green-Blue) coding) into a coding TLS, known in itself, builds around the three parameters constituted by the Complexion, the Luminance and the Saturation of the image. The TLS coding makes it possible in particular to calculate the average saturation Sm of the image. This quantity is particularly relevant to the invention since it makes it possible to segment the location method in three possible cases as a function of the value of the average saturation Sm, each of these cases being optimized in number of parameters and in calculation time. Indeed, the basic idea for the search of the initial approach contour is to apply a so-called flesh color filter on the Tint component of the TLS coded image, so as to show at least the face of the person to extract. However, the degree of relevance of the flesh color filter depends on the average saturation Sm of the image; the greater the saturation, the more the application of the flesh color filter will lead to good results in terms of definition of the approximate initial contour. In practice, the invention sets two thresholds S1 and S2> S1 for the average saturation Sm: - if the saturation Sm is high, greater than S2, we find ourselves in a case where the screening of flesh tint is sufficient on its own to precisely locate the position in the image of the person to be extracted, - if the saturation Sm is average, ie between SI and S2, the flesh color filtering can not be used alone to give an approximate location acceptable to the person. In this case, luminous histogram equalization is used on the Luminance component of the TLS coding. This operation has the effect of eliminating shadows, without significant influence on the characteristics of the image, which makes the filtering shade flesh more relevant and more meaningful. If necessary, the results can be further improved by applying a complementary segmentation by the movement of the image on the assumption that in the videophone images the subject of interest is part of the image elements that can move; in fixed video telephony it is usually the only moving element, while in mobile videophone other elements may be moving such as moving leaves or passing cars. Insofar as, in this case, a segmentation by the very precise image movement is not necessary, segmentation is used by simple difference of images. This method is explained in detail in the article Localization of people in a video sequence (A. Rojbi, J.-C. Schmitt, A. Georges, P. Boissanade, January 17, 2003, University Claude-Bernard Lyonl).

  if the saturation Sm is small, less than st the flesh-tone filtering is no longer significant, then a luminance histogram equalization, followed this time by a luminance histogram thresholding consisting of calculate the luminance gradient to determine the transition zones and the contours of objects. In this case, the motion segmentation is performed by another method known as the maximum likelihood, much more accurate than the simple difference of images. This method is also described in the article Locating People in a Video Sequence Above.

  Finally, it can be seen in FIG. 2 that a human form model is applied to the segmented image obtained at the end of any one of the three possible treatments in relation to the value of the saturation Sm. The result of this operation is the determination of the approximate initial contour sought for the person to be extracted and the elimination of the objects retained by the segmentation by the movement but which are not persons. The application of the shape model is also described in the article Locating people in a video sequence mentioned above.

  The values of the thresholds S1 and S2 of saturation are determined experimentally. The applicant has established that values of 0.4 for 5 SI and 0.7 for S2 lead to satisfactory results.

  Returning to FIG. 1, it can be seen that after the initialization step A is carried out a step B aimed at choosing the most effective active contour algorithm in terms of calculation time and parameterization to arrive at the final contour of FIG. the person to be extracted starting from the initial contour approximated io previously obtained.

  As shown in FIG. 1, the choice of an active contour algorithm takes place between two possibilities, namely, on the one hand, a robust algorithm, the B-snake-GVF algorithm, which can converge very rapidly towards the final contour even starting from an approximate initial approach distant, but unusable in the presence of several people in the image, and, on the other hand, a less robust algorithm, the B-snake algorithm, which nevertheless presents the advantage of being able to be used even if several people are present in the image.

  To choose between these two algorithms, it is necessary to determine the number NBP of people participating in the video image to be analyzed (B1). If NBP = 1, then one will use the algorithm B-snake-GVF (B3), on the other hand, if NBP> 1 one will prefer the algorithm B-snake (B2).

  Figure 3 shows how the NBP number can be determined. From the luminance histogram of the image, the presence or absence of one or more valleys is identified. The term valley here means a global minimum gray level area defined from the luminance histogram. If the image does not include such valleys, it is concluded that only one person can be in the picture, hence NBP = 1.

  In the case of detection of one or more valleys, the corresponding separation points define areas potentially representing people or objects. To determine whether they are people or objects, the areas thus obtained are applied to a classical human form model by correlating these areas with the shape model. A 20% error in the correlation is accepted to hold the area as a person's. In the case where the points of separation highlighted are not validated as areas representing people but rather objects, these areas are not retained. The NBP number of people is then that of the remaining areas selected and validated.

  Finally, the active contour algorithm thus defined from the NBP number is applied to the approximate initial contour to obtain at the output of step B the final contour of the person to be extracted.

  Step C of FIG. 1 corresponds to the implementation of the fuzzy segmentation method itself, which uses the final contour obtained at the end of step B of the method of localization of persons which has just been described. .

  As can be seen in Figure 1, the fuzzy segmentation, which is to blur the background of the image to keep only the person of interest, involves a confidence index IC between 0 and 1. This IC index is used when choosing between a simple fuzzy segmentation of the image consisting of multiplying the inside and the outside of the final contour by 1 and 0 respectively, and a complex fuzzy segmentation in which the inside of the final contour is multiplied by 1 while the outside of this contour is multiplied by a number between 0 and 1 provided by a Gaussian filter, this to take into account a possible uncertainty in the definition of the final contour .

  In general, this index IC is expressed in the following manner: IC = λ * Sm (1 ×) * B × × where Sm is the average saturation of the image, Bm is the average value of the noise in the image , x a parameter characteristic of a rotation movement in the image equal to 0.08 in the presence of a rotation movement and 0 in the absence of such a movement, and a characteristic parameter of the texture of the image 0.06 for a textured image and 0 for a non-textured image (an image will be textured if it has more than 12 homogeneous regions).

  Experimentally, X is taken as 0.9, which shows that saturation Sm is the most significant parameter to consider in the previous relationship.

  The confidence index IC thus defined is then compared (Cl) with a threshold 5 ICs itself determined experimentally, for example 0.52.

  If the localization method uses only one active contour algorithm (B-snake for example), we will use a simple segmentation if IC≥ICs or a complex segmentation if IC <ICs.

  If, according to FIG. 1, two active contour algorithms are used according to the value of the NBP parameter, then the choice of the segmentation to be applied depends on the two parameters IC and NBP. For IC> = ICs or if IC <ICs with NBP = 1, a simple fuzzy segmentation (C2) is performed, whereas for IC <ICs with NBP> 1, a complex fuzzy segmentation (C3) is performed.

  As a numerical example, we can take in the equation of IC Sm = 0.8, Bm = 0.1, with the presence of a rotational movement and a very textured image, which gives IC = 0 57> ICs = 0.52. We will use in this case a simple fuzzy segmentation regardless of the NBP number.

  Finally, a last step designated by D in FIG. 1 is to perform a processing of the segmentation data resulting from step C. This processing is not only intended to follow the temporal evolution of the final contour, but also to detect sudden changes in the video image. In this case, the automatic initialization step A is taken over from the beginning in order to reconstruct the approximate initial contour and obtain the final contour by applying the appropriate active contour algorithm.

  It can also be seen in FIG. 1 that the data processing allows a periodic calculation of the confidence index CI, for example all the N images with N possibly equaling 30.

  FIG. 4 illustrates two possible non-limiting applications of the fuzzy localization and segmentation method according to the invention in the field of video telephony.

  The first concerns mobile telephony where we try to eliminate from the image all unnecessary complex elements from the bottom of the image in order to increase the compression and simplify the video coding of the transmitter side and the video decompression of the side. of the receiver.

  The second concerns residential video calling, where the aim is more to suppress the background of the image in order to preserve the intimacy of the transmitter by fuzzy segmentation performed at the receiver.

  In any case, the method of the invention is implemented by a computer program comprising code instructions recorded on a support readable by a software module installed downstream of the camera on the side of the camera. transmitter and, symmetrically, on the receiver side.

Claims (13)

  A method of locating a person in a color coded video image according to a given color coding system, said method using at least one active contour algorithm for determining a final contour of the person to be extracted from a approximated initial contour, characterized in that said method comprises the following steps: -a) determining an approximate initial contour of the person to be extracted, consisting of: - a1) transforming the color coding system of the image into a coding TSL, - a2) perform an initial segmentation of the image from the calculation of the average saturation Sm of the image and its comparison with two predetermined thresholds SI and S2 (> SI): -a21) if Sm> S2 : apply to the image a flesh tint filter, -a22) if SI <Sm <S2: apply to the image a histogram equalization operation, then a flesh tint filter, -a23) if Sm <S1 : Apply a histogram equalization operation to the image followed by a histogram thresholding operation, then a first segmentation operation by the movement of the image, -a3) applying a shape model to the image thus segmented and deducing from it the approximate initial contour; b) selection of an active contour algorithm, -c) application of said active contour algorithm to the approximate initial contour to obtain the final contour locating the person to extract in the video image.   2. Location method according to claim 1, characterized in that said method also comprises a data processing step for determining the temporal evolution of the final contour so as to update it image by image.   3. Location method according to one of claims 1 or 2, characterized in that step a) is resumed in the case of a sudden change in the video image.   4. Location method according to any one of claims 1 to 3, characterized in that step a22) is completed by a second segmentation operation by the movement of the image.   5. Location method according to one of claims 1 to 4, characterized in that said first segmentation operation by the movement of the image is a maximum likelihood method.   io 6. Location method according to one of claims 4 or 5, characterized in that said second segmentation operation by the movement of the image is a simple difference image method.   7. A localization method according to any one of claims 1 to 6, characterized in that the active contour algorithm chosen in step b) is the B-snake algorithm.   8. Location method according to any one of claims 1 to 6, characterized in that the active contour algorithm chosen in step b) is the B-snake-GVF algorithm.   9. A method of locating according to any one of claims 1 to 6, characterized in that step b) of choosing an active contour algorithm comprises the following steps: -b1) determination of the NBP number of people in the video image, -b2) choice of the algorithm: -b21) if NBP = 1: choice of the algorithm B-snake-GVF, -b22) if NBP> 1: choice of the algorithm B-snake.   10. A method of fuzzy segmentation of a person in a video image, characterized in that said method consists of locating said person by means of the localization method according to any one of claims 1 to 8, then to perform the segmentation operations fuzzy: -dl) compute a confidence index CI, -d2) compare said confidence index CI to a predetermined threshold ICs: -d21) if IC≥ICs: perform a simple segmentation of the image by multiplying by 1 and 0 inside and outside the final contour, -d22) if IC <ICs: perform a complex segmentation of the image by multiplying by 1 the inside of the final contour and by a number between 0 and 1 provided by a Gaussian filter outside said contour.   11. A method of fuzzy segmentation of a person in a video image, characterized in that said method consists of locating said person by means of the locating method according to claim 9, then performing the following fuzzy segmentation operations: -dl) calculating a confidence index CI, -d2) comparing said confidence index CI with a predetermined threshold ICs: -d21) if IC≥ICs and if IC <ICs with NBP = 1: performing a simple segmentation of the image by respectively multiplying by 1 and 0 inside and outside the final contour, -d22) if IC <ICs with NBP> 1: perform a complex segmentation of the image by multiplying by 1 the inside of the final contour and by a number between 0 and 1 provided by a Gaussian filter outside said contour.   12. Segmentation method according to one of claims 10 or 11, characterized in that the confidence index IC is calculated every N images. A computer program product comprising program code instructions recorded on a computer readable medium, for carrying out the steps of the method according to claims 1 to 12 when said program is executed by a computer.