FR2892846A1 - Audio segment representations similarity measurement calculating method for speaker tracking, involves attributing weighting factor, representing intraclass similarity measurement between representations, to basic similarity measurement - Google Patents

Abstract

The method involves determining a weighting factor for each audio segment of a class to which an audio segment to be tested belongs. The factor represents an intraclass similarity measurement between representations of the segment to be tested and the segment. The determined factor is attributed to a basic measurement of similarity of the relating segment. A measurement of similarity is obtained from the weighted basic measurements. The intraclass measurement is obtained by applying a similarity measurement calculation function different from that utilized for basic measurement calculation. Independent claims are also included for the following: (1) a reference speaker tracking method (2) a similarity measurement calculating device (3) a reference speaker tracking device comprising the similarity measurement calculating device (4) a computer program comprising program instructions adapted to the implementation of a similarity measurement calculating method.

Description

The present invention relates to a method for calculating the measurement of

  similarity between a reference audio segment and an audio segment to be tested, particularly during an application in a method of tracking a reference speaker in an audio document. To compare and measure similarities between audio segments, several techniques exist. For example, a measure of similarity between two representations of speech segments X and Y can be calculated using modeling by the anchoring models as explained in the document by D, Sturim; D, Reynolds; E, Singer and J, Campbell, "speaker indexing in large audio databases using anchor models" (ICASSP2001, pp. 429-432). In this type of calculation, the confidence in the similarity measure is even lower when the X and Y segments are short. To overcome these drawbacks and increase the reliability of a measurement of similarity on a segment X which can be short, it is known to list the audio segments Y, for example from the same audio document, in segments classes, each class grouping Yk segments closest in terms of similarity. These close segments preferably come from the same speaker. Thus, the measure of similarity between a representation (X) of a reference segment X and a representation () of a segment Y to be tested belonging to a class C comprising k segments is performed by calculating a mean of all measures of similarity between and representations of the Yk segments of the class. This is for example formulated as a possibility of obtaining measurement of distance between two segments in the document entitled "Speaker tracking in broadcast audio material in the framework of the THISL project" proposed by Couvreur, L and Boite, JM (Proc.of the ESCA ETRW Workshop Accessing Spoken Audio Information, pages 84-89, 1999). This document also describes a way to build such classes and to automatically list the audio segments, for example from an audio document, in their respective class. - 2

  A disadvantage of such a technique is that this type of automatic segment class construction generates groupings of non-homogeneous segments from, for example, different speakers. A similarity measure calculation based on the use of such classes, as previously mentioned, will then be imprecise. The non-homogeneity of the segments of the same class has a significant influence on the calculation of a measure of similarity of an element of this class. The aim of the invention is to overcome these drawbacks by proposing a similarity measurement calculation method that uses the advantages of a calculation based on the use of segment classes while limiting the influence of the non-homogeneity of the segments. within a class. For this purpose, the invention proposes a method for calculating a measure of similarity between a representation of a reference audio segment and a representation of an audio segment to be tested, the audio segment to be tested belonging to a class comprising a plurality of audio segments, the similarity measure being obtained from so-called elementary similarity measurements between the reference segment and each of the segments of said class. The method according to the invention comprises the following steps: determining a weighting factor for each of the segments of said class; assigning the determined weighting factor to the elementary similarity measure of the corresponding segment; and obtaining the similarity measure from the elementary similarity measures thus weighted.

  Thus, the assignment of a weighting factor adapted to each segment of the class makes it possible to differentiate the influence of the segments on the calculation of the similarity measure between a representation of a reference segment and a representation of a segment. to be tested belonging to the class. In a preferred embodiment, the weighting factor for a given segment of said class is representative of a so-called intraclass similarity measurement between a representation of the audio segment to be tested and a representation of said segment. Thus, the influence of a segment of the class is more or less important depending on whether this segment is close to the segment to be tested.

  Advantageously, the weighting factor is of low value when the measurement of intraclass similarity is low and high otherwise. In a particular embodiment, the intraclass similarity measurement is obtained by applying a similarity measure calculation function different from that used for the elementary similarity measurement calculation.

  This has the advantage of adapting the measurement of similarity measurement to the environment in which the segments are located and thus optimize the process. In another particular embodiment, for the sake of simplification, the intraclass similarity measurement is obtained by applying a similarity measure calculation function identical to that used for the elementary similarity measurement calculation. In an alternative embodiment, the reference segment belongs to a reference class comprising a plurality of reference audio segments and the method further comprises a step of assigning a second weighting factor for each of the reference segments of the reference segment. reference class.

  Preferably, the second weighting factor for a given reference segment is a function of a similarity measure between a representation of a current reference segment and a representation of said given reference segment, the segments belonging to the class of reference. reference. The invention also relates to a method for tracking a reference speaker in an audio document, which comprises the following steps: - increasing the document to be tested into a plurality of audio segments; assigning each of the segments resulting from the segmentation step to a segment class; for each segment resulting from the segmentation: -4-

  calculating the similarity measure between a representation of a segment corresponding to the reference speaker and a representation of the current segment of the segmentation by the implementation of a method as described above; decision on the recognition of the reference speaker for the current segment by comparison with a predetermined threshold of the corresponding similarity measure resulting from the calculation. The use of the similarity measurement calculation method according to the invention in a speaker tracking method thus increases the accuracy of the result derived from this speaker tracking method.

  The invention also relates to a similarity measuring calculation device between a representation of a reference audio segment and a representation of an audio segment to be tested, the audio segment to be tested belonging to a class comprising a plurality of audio segments. The device comprises means for calculating so-called elementary similarity measurements between the reference segment and each of the segments of said class, and further comprises: means for determining a weighting factor for each of the segments of said class; means for assigning the weighting factor derived from the determination means to the elementary similarity measurement of the corresponding segment; and means for obtaining the similarity measure from the weighted elementary similarity measurements. The invention also provides a device for tracking a reference speaker in an audio document. This device comprises: means for segmenting the document to be tested into a plurality of audio segments; means for assigning each of the segments coming from the segmentation means to a class of segments; a device for calculating the similarity measure between a representation of a segment corresponding to the reference speaker and a representation of a -5-

  segment from the segmentation means, as described above, the device being applied to each segment from the segmentation means; means for deciding on the recognition of the reference speaker applied to each of the segments coming from the segmentation means, comprising means of comparison with a predetermined threshold of the corresponding similarity measure coming from the similarity measure calculation device. These devices implement the similarity measurement and speaker tracking calculation methods respectively.

  The invention also relates to a computer program comprising program instructions adapted to the implementation of a similarity measurement calculation method according to the invention as described above and / or a speaker tracking method. as described above, when said program is loaded and executed in a computer system.

  Finally, the invention aims at a means of storage, possibly completely or partially removable, readable by a computer, storing a set of instructions executable by said computer to implement the similarity measuring method and / or the method of monitoring speaker according to the invention. Other features and advantages of the invention will emerge more clearly on reading the following description, given solely by way of nonlimiting example, and with reference to the appended drawings, in which: FIG. 1 represents an embodiment a device embodying the invention; FIG. 2 illustrates the main constituents of the constituent elements of the similarity measurement and speaker tracking calculation device according to the invention; FIG. 3 represents a flowchart illustrating the main steps of the similarity measure calculation method according to the invention; and -6

  FIG. 4 represents a flowchart illustrating the main steps of the speaker tracking method according to the invention.

  According to an embodiment chosen and shown in FIG. 1, a device embodying the invention is for example a microcomputer 10 which comprises, in a known manner, in particular a processing unit 12 equipped with a microprocessor, a read-only memory type ROM 13, RAM 14. The microcomputer 10 may include in a conventional and non-exhaustive manner the following elements: a keyboard, a screen, a microphone, a speaker, a communication interface, a disk drive, storage means ... The read-only memory 13 comprises registers storing a computer program PG1 comprising program instructions adapted to implement a method of calculating a similarity measure according to the invention such that as described later with reference to FIG. 3. This program PG1 is thus adapted to calculate a similarity measure between a representation of a reference segment X that the processing unit ent receives as input 15 and a representation of a segment Y that the processing unit receives input 11 via an audio document to be tested for example.

  The read-only memory 13 also comprises a program PG2 comprising program instructions adapted to implement a speaker tracking method as described with reference to FIG. 4. This program PG2 is adapted to perform a speaker follow-up corresponding to the speaker of the speaker. reference represented by the input X segment 15 of the processing module, in an audio document ii provided at the input of the processing unit. The processing unit implementing the program PG2 outputs 16, the segments corresponding to the reference speaker that have been detected in the audio document to be tested 11. These inputs, segment X 15 and audio document 11, can come from an external database (via a computer network for example) or internal to the device.

  When the power is turned on, the programs PG1 and PG2 stored in the read-only memory 13 are transferred into the random access memory which will then contain the executable code of the invention as well as registers for storing the variables necessary for the implementation of the 'invention.

  More generally, a means of storage, readable by a computer or by a microprocessor, integrated or not integrated into the device, possibly removable, stores a program implementing the similarity measurement calculation method according to the invention and / or a program implementing the speaker tracking method according to the invention.

  Referring to Figure 2, we will describe the constituent elements of a speaker tracking device according to the invention, incorporating a similarity measuring calculation device according to the invention. The similarity calculating device may be independent or integrated into other devices requiring this type of similarity measurement.

  The device thus described is for example integrated in a computer or sound processing apparatus. The speaker tracking device 20 includes a segmentation module 22 of an audio document. An audio document to be tested 21, for example from an audio document database, is provided to the segmentation module 22 which includes segmentation means adapted to segment this audio document into a plurality of audio segments Y 1, i ranging from 1 to N. A class assignment module 23 comprises allocation means able to assign each of the segments Yi coming from the segmentation module to a class CY, segments comprising a plurality of similar segments in a similarity grid. A class CYi comprises, for example, the segments Yk, k ranging from 1 to K. A similarity measurement calculating device 25, which will be described later, is suitable for implementing the similarity measurement calculation method according to FIG. 'invention. A weighted calculation is performed by the device 25, from a reference segment X, referenced 24 in FIG. 2, provided at the input of this device and which comes from a reference speaker. At the output of this device, a decision module 29 comprises decision means able to compare the similarity measure received from the computing device 25 with a predetermined threshold in order to decide on the similarity of (representation of the segment Y; (representation of the reference segment X). This decision module outputs, a set of segments 30 which are considered to belong to the reference speaker. The similarity measure calculation device 25 comprises an intraclass similarity measurement calculation module 26 including similarity measure calculation means adapted to calculate a measure of intraclass similarity between a representation () of a test segment Y; current belonging to a class Cy, and a representation (Y;) of a segment Yk of the same class. This calculation is made from a segment to be tested Y; current, for all Yk segments of the same class. A module for assigning a weighting factor comprises means for determining a weighting factor Pik as a function of the intraclass similarity measurements received at the output of the module 26 and of the means for assigning the weighting factor thus determined to a measure of elementary similarity between a representation (X) of the reference segment X and a representation (Y) of a segment Yk of the class Cy, which contains the segment Yi to be tested. At the output of the module 27, there is a module 28 for obtaining the similarity measure between a representation of the reference segment X and a representation of the segment to be tested 20, from the weighted elementary similarity measurements. This module 28 comprises weighted calculation means capable of taking into account the weighting factors associated with the elementary similarity measurements. This similarity measure can be performed for all the test segments Y, contained in the audio document 21, so that a decision is made by the decision module 29 for all segments of the audio document. The main steps of a similarity measurement calculation method according to the invention will now be described with reference to FIG. The similarity measure calculation method between a representation of a reference audio segment X and a representation of an audio test segment Y according to the invention is implemented by the similarity measure calculation device 25 described. with reference to FIG. 2. From a test audio segment Y, belonging to a segment class Cy, grouping K segments Yk, a preliminary step 32 calculates a measurement of intraclass similarity between a representation of the segment to be tested Y, belonging to the class Cy, and a representation of each of the segments Yk belonging to the same class Cy ,. A representation of a segment is for example obtained after an acoustic analysis step and a specific processing step. One can for example use an anchor model representation as described in the document entitled "speaker indexing in large audio databases using anchor models" D authors, Sturim; D, Reynolds, E, Singer and J Campbell (ICASSP2001, pages 429-432). Step 33 is a step of obtaining a weighting factor plk for each of the Yk segments of the class Cy, in which the segment to be tested Y, is included. This weighting factor is according to a preferred embodiment according to an intraclass similarity measurement obtained in the prior step 32. The step 33 is followed by the step 34 where the weighting factor p1k is assigned to a measurement. elementary similarity between a representation of the reference segment X referenced here at 35 and from a reference speaker and a representation of a segment Yk of the class Cy ,. This allocation makes it possible to obtain, in step 36, weighted elementary similarity measurements which will make it possible to obtain the similarity measure between a representation of a segment of a fair market and a segment of the market. test by a function & mu according to equation (1) of t pe. - 10 -

  where p, k is a function, as previously described, of the intraclass similarity measure 8 (Y,., YX). This weighting factor can be defined for example by the equation (2): P, k = + tanh (5 (0.5 -8 (f, t))) (2) In a particular embodiment, the calculation functions similarity measures d and é are the same. This has the advantage of using common means and thus simplify the implementation. These similarity measures are defined for example by a measure of similarity of the type of correlation similarity measure p proposed by the authors Collet, M; Charlet, D; and Bimbot, F in "IEEE International Conference on Acoustics, Speech and Signal Processing, 2005". In another particular embodiment, the calculation functions d and 8 are different.

  Thus, the first elementary similarity measurement calculation function between representations of audio segments from different audio documents, and therefore with different acoustic conditions, must be robust to variations in the acoustic environment. Similarly, a representation of the segments used for this calculation can be advantageously obtained by an acoustic analysis step that is robust to variations in the acoustic environment. The intra-class similarity measurement calculation function does not need to be robust to variations in the acoustic environment. Likewise, the acoustic analysis used to obtain a representation of the segment does not need to be robust to acoustic environment variations. Thus, the use of different functions for obtaining elementary and intraclass similarity measurement, or even the use of different segment representations for each of the measurements, has the advantage of obtaining adapted and optimal measurements. The first function, d can for example be based on a correlation function as mentioned above, the second function can be a function using an angular measure as mentioned in the same document "A correlation metric for speaker tracking using anchor models" (IEEE International Conference on Acoustics, Speech and Signal Processing, 2005). The method of calculating a measure of similarity between a representation of a reference segment X and a representation of a segment to be tested Y; ends with obtaining this measure of similarity dp (X, Y). In this exemplary embodiment, it has been assumed that only one reference segment X from a reference speaker. In an alternative embodiment, the similarity calculation method can be carried out between two classes, the reference segment X then belonging to a reference class named Cx comprising Kx segments and the segment to be tested Y belonging to a class named Cy comprising Ky segments.

  In this variant, the method determines a first weighting factor pjk, a function of a measure of intraclass similarity of Cy between representations of segments Yj and Yk pertaining to the class Cy and a second weighting factor pik, a function of a measurement. of intra-class similarity of Cx between representations of reference segments Xi and Xk pertaining to the reference class Cx.

  The weighted calculation is then performed according to the following equation (4): K ((CX, CY) ù ld (X,., YJ.) (4) tzr J.ca. 1 = 1 j = 1 1 = 1 t = 1 We will now describe, with reference to FIG 4, the speaker tracking method according to the invention.In step 42, a segmentation in speakers is performed on the audio document to be tested 41. This segmentation allows to cut the audio document to test nk, Ky 1 p ~~ 1 Pi * where at = and aJ = K, KY k = 1 - 12 -

  in homogeneous segments, that is to say uttered by the same speaker. A segmentation method is proposed by the authors P.Delacourt and CJ; Wellekens in a document entitled "Segmentation in speakers of an audio document" (CORESA99: 5th days of studies and exchanges COmpression and Representation of Audiovisual Signals, Sophia Antipolis1999). At the end of step 42, a plurality of test audio segments Y ,, i ranging from 1 to N are obtained. Step 42 is followed by step 43 of assigning segments Y to a class of Cy segments ,. This class assignment step, also called clustering, consists initially of grouping a segment and its nearest neighbor together in the same class. In a second step, we merge classes that have an empty intersection. We thus obtain a set of classes in which are distributed Y segments, from the segmentation. Step 43 is followed by the weighted calculation step 45 implementing the similarity measurement calculation method described with reference to FIG. 3. This weighted calculation is performed with respect to a reference segment X, referenced 44. , from a reference speaker that we want to find in the document to be tested. At the end of step 45, a measure of similarity is obtained between a representation of the reference segment X and a representation of the current segment Y, originating from the audio document. This weighted calculation step is performed for all the audio segments to be tested Y belonging to the audio document to be tested. All these similarity measures are then compared to a threshold 0 in step 45. The threshold 0 may for example have a value of 0.5. If the similarity measure d (X, Y.) Is less than the threshold A, then a decision is made as to the membership of the segment Y, to the reference speaker. All segments thus detected as audio segments uttered by the reference tutor are grouped together and this completes the tracking method of the Iocutcur.

Claims (12)

  A method of calculating a measure of similarity between a representation of a reference audio segment and a representation of an audio segment to be tested, the audio segment to be tested belonging to a class comprising a plurality of audio segments, the measurement similarity being obtained from so-called elementary similarity measurements between the reference segment and each of the segments of said class, characterized in that it comprises the following steps: determining a weighting factor for each of the segments of said class ; assignment of the weighting factor undetected to the elementary similarity measure of the corresponding segment; and obtaining the similarity measure from the elementary similarity measures thus weighted.   2. Method according to claim 1, characterized in that the weighting factor for a given segment of said class is representative of a so-called intraclass similarity measurement between a representation of the audio segment to be tested and a representation of said segment.   3. Method according to claim 2, characterized in that the weighting factor is of low value when the measurement of intraclass similarity is low and high otherwise.   A method according to claim 2 or 3, characterized in that the intraclass similarity measure is obtained by applying a similarity measure calculation function different from that used for the elementary similarity measure calculation.   5. Method according to claim 2 or 3, characterized in that the measurement of intraclass similarity is obtained by the application of a similarity measurement calculation function identical to that used for the calculation of elementary similarity measurement. -   6. Method according to one of claims 1 to 5, characterized in that the reference segment belongs to a reference class comprising a plurality of reference audio segments and in that it further comprises a step of assigning a second weighting factor for each of the reference segments of the reference class.   7. Method according to claim 6, characterized in that the second weighting factor for a given reference segment is a function of a similarity measure between a representation of a current reference segment and a representation of said given reference segment. , the segments belonging to the reference class.   8. A method of tracking a reference speaker in an audio document, characterized in that it comprises the following steps: segmentation of the document to be tested into a plurality of audio segments; assigning each of the segments resulting from the segmentation step to a class of segments; for each segment resulting from the segmentation: calculating the similarity measure between a representation of a segment corresponding to the reference speaker and a representation of the current segment of the segmentation by the implementation of a method according to the one of claims 1 to 7; and - decision on the recognition of the reference speaker for the current segment by comparison with a predetermined threshold of the corresponding similarity measure from the calculation. measuring lari   9. Device for calculating a similarity between a representation of a reference audio segment and a representation of an audio segment to be tested, the audio segment to be tested belonging to a class comprising a plurality of audio segments, the device comprising means for calculating so-called elementary similarity measurements between the reference segment and each segment of said class, characterized in that it further comprises: means for determining a weighting factor for each of the segments of said class; means for assigning the weighting factor derived from the determination means to the elementary similarity measurement of the corresponding segment; and means for obtaining the similarity measure from the weighted elementary similarity measurements.   10. Device according to claim 9, characterized in that it further comprises so-called intraclass similarity measurement calculation means between the audio segment to be tested and each of the segments of said class.   11. Device for tracking a reference speaker in an audio document, characterized in that it comprises: means for segmenting the document to be tested into a plurality of audio segments; means for assigning each of the segments coming from the segmentation means to a class of segments; a device for calculating the similarity measure between a representation of a segment corresponding to the reference speaker and a representation of a segment coming from the segmentation means, confounds with one of the claims 9 to 10, the device being applied to each of the segments from the segmentation means; and decision means for the recognition of the reference speaker applied to each of the segments from the segmentation means, comprising means for comparing with a predetermined threshold of the corresponding similarity measure from the similarity measure calculation device.   A computer program comprising program instructions adapted to implement a similarity measure calculation method according to any of claims 1 to 7 and / or a speaker tracking method according to the present invention. claim 8, when said program is loaded and executed in a computer system.