EP4290514A1 - Speaker voice masking - Google Patents

Abstract

The invention relates to masking a speaker's voice by intentionally altering the pitch and timbre of the voice. An audio signal corresponding to an original recording of the speaker's voice is divided (11) into a series of successive audio segments of determined constant duration. An ascending frequency alteration (12a) is applied to a timbre (A) extracted from each audio segment. A descending frequency alteration (12b) is applied to a pitch (B) extracted from the audio segment. The altered pitch and altered timbre of the audio segment are combined (14) to form a single resulting altered audio segment. From one audio segment to another in the series of audio segments, a variation (13a) of the ascending alteration and a variation (13b) of the descending alteration is applied. These variations fluctuate randomly from one audio segment to another in the series of audio segments.

Description

Technical area

The present invention relates to masking the voice of a speaker, in particular to protect the identity of the speaker by restricting the possibility of identifying him by analysis of an original recording of his voice.

It finds applications, in particular, in audio-phonic or audio-visual editing and/or mixing systems in which it can be implemented by audio-phonic processing software.

Technology background

In certain areas of the audiovisual industry, for example, it is useful to be able to broadcast programs (audio, and/or video, and/or multimedia content), while concealing the identity of the speaker in order to protect the latter from any type of consequences of this diffusion which may be damaging for him. For example, in investigative journalism, it is common to anonymize the recording of a witness's interview which could be used against the witness's interests, either by the perpetrators of crimes he or she denounces, either by litigants or by a competent judicial institution if the witness admits to having violated any regulation.

Techniques aimed at anonymizing the voice of a speaker in an audio signal, that is to say making it difficult to identify the speaker from an analysis of the audio signal, have been known for a long time. The oldest and most widespread simply transforms the speaker's voice by simply shifting harmonics. This shift can be achieved either towards high frequencies, that is to say towards the treble, or towards low frequencies, that is to say towards the bass. In reference to characters from fictional audiovisual programs well known to the general public, it is sometimes said that the voice thus transformed resembles the voice of “Mickey-Mouse” ^™ or the voice of “Darth Vader” ^™ , which are generated by such techniques from the voice of a real person. However, the transformation of the voice thus obtained is easily reversible with technical means. today accessible by many people, if not by everyone.

Furthermore, voice recognition software, or more particularly speaker recognition software based on their voice signature, is sometimes used by the police to identify people making telephone or other threats. the origin of anonymous calls. However, there is today certain software of this type which makes it possible to identify a speaker with a level of reliability such that it can lead to a person being convicted by the courts. Even if such use may seem laudable from the point of view of the community, malicious uses of such software can have consequences for individuals who are much less so, sometimes creating irremediable harm such as in the case of harm to life. private. This is why audio-phonic processing techniques can be used to protect speakers whose voice recording is likely to be broadcast or intercepted on communication networks.

Finally, another case where protection of speakers is desirable is that of voice input applications using speech recognition to allow access to services by users. Speech recognition is used to recognize what is being said. Therefore, it helps transform speech into text and that is why it is also known as speech to text conversion.

The article “Voice Mask: Anonymize and Sanitize Voice Input on Mobile Devices” published in the scientific journal COMPUTER SCIENCE, CRYPTOGRAPHY AND SECURITY, Cornell University, US, November 30, 2017, pages 1-10, Jianwei Qian et al., thus disclose that with hands-free communication, voice input has largely replaced the use of traditional keypads (for example, Google ^® , Microsoft ^® , Sougou ^™ and iFlytek ^™ virtual keyboards). These techniques are used daily by many users for voice search (with, for example, applications like Microsoft Bing ^® , Google Search ^® ) and personal assistants based on artificial intelligence (e.g. Siri ^® from Apple ^® , and Amazon Echo ^® ), with a wide range of mobile devices. In these applications, due to resource limitations on mobile devices, the speech recognition operation is generally offloaded to a cloud computing server for greater accuracy and efficiency. It follows that the privacy of users may be compromised. Indeed, even if in these applications only the speech content needs to be recognized by speech recognition, it has become easy to practice speaker recognition in order to recognize usual mobile users by their voice, via learning techniques taking advantage of the recurrence of uses, to analyze the sensitive content of their inputs via speech recognition, and then to draw up their user profile on the basis of this content in order to provide biased responses to their requests and/or making them targeted commercial proposals. The authors of the article propose a voice neutralization application that provides good protection of the user's identity and private speech content, at the cost of minimal degradation of the quality of voice recognition. It adopts a voice conversion mechanism that resists multiple attacks.

The article “Speaker Anonymization for Personal Information Protection Using Voice conversion Techniques”, Proceedings of Access 2020, Digital Object Identifier, Vol.8 2020, pages 198637-198645, IEEE, US, In-Chul Yoo et al., discloses voice conversion to ensure speaker anonymization with the objective of retaining the linguistic content of the given speech while removing biometric data from the original speaker's voice. The proposed method modifies conventional speaker identity vectors into anonymized speaker identity vectors using various methods.

The article titled “Speaker Anonymization Using X-vector and Neural Waveform Models”, Proceedings of 10th ISCA Speech Synthesis Workshop, September 20-22, 2019, Vienna, Austria, pages 155-160, Fuming Fan et al., propose a speaker anonymization approach to conceal the identity of the speaker while maintaining a high quality of anonymous speech, which is based on the idea of extracting linguistic and speaker identity features from an utterance and then using them with acoustic and waveform neural models to synthesize anonymous speech. The original identity of the speaker, in the form of the timbre, is removed and replaced with that of an anonymous pseudo-identity. The approach exploits speakers' state-of-the-art representations in the form of X-vectors. These representations are used to derive anonymous speeches. These are used to derive pseudo-identities of anonymous speakers by combining several vectors X of random speakers.

The authors of the article entitled “Exploring the Importance of FO Trajectories for Speaker Anonymization using x-vectors and Neural Waveform Models”, International Audio Laboratories, Erlangen, 2021, Workshop on Machine Learning in Speech and Language Processing (MLSLP), September 6, 2021, ISCA, DE, pages 1-6, EU Gaznepoglu et al., considering the presence of personal information in the different components of the fundamental frequency F0 of a speaker's voice and the availability of various approaches to modify the F0 component, propose an exploration of their potential in the context of the voice anonymization. They suggest that decomposing the F0 component, modifying speaker-related features, possibly disrupting with noise during the process, and then resynthesizing, could increase anonymization performance and/or improve intelligibility. It is mentioned that the approaches proposed so far, such as shifting and scaling, all depend on the identity of the person to be protected.

The article “Speaker anonymization using the McAdams coefficients” in the journal COMPUTER SCIENCE, AUDIO AND SPEECH PROCESSING, Cornell University, US, September 2021, pages 1-5, Patino J et al., discusses the reversibility of anonymization. The authors present their work to further explore the potential of well-known signal processing techniques as a solution to the anonymization problem, as opposed to other more complex and demanding solutions that require training data. . They suggest optimizing an original solution based on McAdams coefficients to modify the envelope spectral (ie, timbre) of speech signals. They sought to confirm that different values of the McAdams coefficient α (alpha), which modify the timbre of the voice, can produce different pseudo-voices for the same speaker. This results in a stochastic approach to anonymization in which the McAdams coefficient is sampled from a uniform distribution range, i.e. α ∈ U(αmin,αmax). However, in the proposed applications, the article is content to teach that the coefficient α can be changed randomly from one speaker to another, indicating that a malicious third party would then have need to know the exact McAdams coefficient used to anonymize the speech of any particular speaker, in order to reverse the transformation.

There remains therefore the need for a technique for masking a speaker's voice that cannot be easily circumvented.

Summary of the invention

• un découpage d'un signal audio correspondant à un enregistrement original de la voix du locuteur en une série de segments audios successifs de durée constante déterminée, et la formation d'une série de couples de segments audios comprenant chacun un primat et un duplicata d'un segment audio de ladite série de segments audios ; ainsi que,

pour chaque couple de segments audios :

• un traitement du primat du segment audio et un traitement du duplicata du segment audio pour en extraire d'une part, un signal caractérisant la hauteur du segment audio, et d'autre part, un signal caractérisant le timbre du segment audio ;
• une première altération, appliquée au signal caractérisant le timbre extrait du segment audio, et ayant pour effet d'altérer tout ou partie de l'enveloppe des harmoniques dudit segment audio , de manière à générer un timbre altéré du segment audio ;
• une deuxième altération appliquée au signal caractérisant la hauteur extrait du segment audio, et ayant pour effet d'altérer la valeur de la fréquence fondamentale, de manière à générer une hauteur altérée du segment audio ; l'une des altérations parmi la première altération et la deuxième altération étant une altération ascendante alors que l'autre altération est une altération descendante ; ainsi que,
• une combinaison du timbre altéré du segment audio et de la hauteur altérée du segment audio, pour former un segment audio altéré résultant,

le procédé comprenant en outre, d'un couple de segments audios à l'autre dans la série des couples de segments audios :

• une variation de la première altération et ;
• une variation de la deuxième altération,

lesdites variations desdites première et deuxième altérations étant fluctuantes aléatoirement d'un couple de segments à l'autre dans la série des couples de segments audios,
et le procédé comprenant en outre :

• la recomposition d'un signal audio masqué à partir de la série des segments audios altérés.

A first aspect of the proposed invention relates to a method of masking the voice of a speaker to protect their identity and/or their privacy by intentional alteration of the pitch and timbre of the voice, comprising:

• a division of an audio signal corresponding to an original recording of the speaker's voice into a series of successive audio segments of determined constant duration, and the formation of a series of pairs of audio segments each comprising a primate and a duplicate of an audio segment of said series of audio segments; as well as,

for each pair of audio segments:

• processing the primacy of the audio segment and processing the duplicate of the audio segment to extract, on the one hand, a signal characterizing the pitch of the audio segment, and on the other hand, a signal characterizing the timbre of the audio segment;
• a first alteration, applied to the signal characterizing the timbre extracted from the audio segment, and having the effect of altering all or part of the envelope of the harmonics of said audio segment, so as to generate an altered timbre of the audio segment;
• a second alteration applied to the signal characterizing the pitch extracted from the audio segment, and having the effect of altering the value of the fundamental frequency, so as to generate an altered pitch of the audio segment; one of the alterations among the first alteration and the second alteration being an ascending alteration while the other alteration is a descending alteration; as well as,
• a combination of the altered timbre of the audio segment and the altered pitch of the audio segment, to form a resulting altered audio segment,

the method further comprising, from one pair of audio segments to another in the series of pairs of audio segments:

• a variation of the first alteration and;
• a variation of the second accidental,

said variations of said first and second alterations being fluctuating randomly from one pair of segments to another in the series of pairs of audio segments,
and the method further comprising:

• the recomposition of a masked audio signal from the series of altered audio segments.

Thanks to this process, the voice is masked, making it possible to meet the requirement for protection of the speaker(s), since the process can easily be implemented in the very first equipment involved in the acquisition and processing chain. audio. At the same time, the process makes it possible to have a final rendering which remains intelligible, that is to say which is neither a voice of “Mickey Mouse” ^™ nor a voice of “Darth Vader” ^™ due to both alterations applied to each audio segment that produce changes in the frequency content that are in opposite directions to each other. In fact, we apply an ascending effect (towards the high tones) for one and a descending effect (towards the low tones) for the other of the two alterations, so that these two effects are combined from the point of view of content frequency of the audio segment considered. The resulting masked audio segment has a frequency content which remains generally closer in spectral dynamics to that of the original audio segment, despite the masking of the voice which is obtained.

Advantageously, the frequency alterations are limited, one always being ascending while the other is always descending. Therefore, the software or device adapted to implement the solution cannot itself do the opposite operation.

According to the method, two components of the spectrum of the audio signal are simultaneously altered compared to the original recording of the speaker's voice. This is a first element favorable to the irreversibility of the method, because a malicious third party wishing to return to the speaker's voice origin will have to play on these two characteristics of the voice in combination, which complicates the task compared to masking by pitch shift alone.

According to another advantage, the variability of alterations is not stationary. It varies over time. Thus there can be several variations over a second of processing.

Finally, the proposed implementation modes provide masking of the voice which is irreversible in audio, that is to say by inverse audio processing.

In addition, the voice masked by the proposed method is non-analyzable by known speaker recognition techniques, and does not expose the speaker to commercial practices violating his privacy using voice recognition techniques, given that the masked voice of the same speaker is never masked in the same way twice.

In advantageous implementation modes, the division of the audio signal into a series of successive audio segments of determined duration can be carried out by temporal windowing independent of the content of the audio signal.

In advantageous embodiments, the division of the audio signal can be configured so that the duration of an audio segment is equal to a fraction of a second, so that successive changes in the parameters varying the first and second alterations occur several times per second.

In advantageous embodiments, the alteration of the pitch of the audio signal corresponds to a variation in the fundamental frequency of the audio signal of any of the following values: ± 6.25%, ± 12.5%, ± 25%, ± 50% and ± 100%.

In advantageous embodiments, the first alteration and the second alteration are dependent on each other, fluctuating jointly so as to satisfy a determined criterion relative to their respective effects on the frequency content of the timbre of the audio segment and on the frequency content of the height of the audio segment, respectively. For example, this criterion may consist of maintaining a minimal gap between the respective effects of the two alterations, and thus avoiding temporarily returning to the original voice.

A second aspect of the invention relates to a computer program comprising instructions which, when the computer program is loaded into the memory of a computer and is executed by a processor of this computer, are adapted to implement implements all the steps of the process according to the first aspect of the invention above.

The computer program for implementing the method can be recorded in a non-transitory manner on a tangible recording medium, readable by a computer.

The computer program for implementing the method can advantageously be sold as a plugin, capable of being integrated into “host” software, for example audio-phonic or audio-production and/or processing software. visual such as Pro Tools ^™ , Media Composer ^™ , Première Pro ^™ or Audition ^™ , among others. This choice is particularly suited to the audiovisual world. In fact, this eliminates the need to transfer the original audio signal (unmasked, therefore in the clear), to a remote server or another computer. It is therefore the user's computer which alone holds the source file of the original voice, that is to say before execution of the masking process. This greatly reduces the risk of malicious interception of the original audio signal. However, the method can be implemented by audio-phonic processing software which can very well be executed on independent hardware equipment and having standard processing capacity, for example a computer for general use. general, because its processing takes place in real time. It does not require the implementation, in particular, of any artificial intelligence, of any voice data bank, nor of any learning process, unlike a number of solutions of the prior art, in particular some of those which were presented in the introduction.

Alternatively, the computer program for implementing the method can advantageously be integrated, either ab initio or through a software update, into the internal software which is embedded in equipment dedicated to production and /or the processing of audio-phonic or audio-visual content (called “media” in the jargon of those skilled in the art), such as an audio and/or video mixing and/or editing console for example. Such equipment is intended more for producers, mixers, and other media post-production professionals.

A third aspect of the invention relates to an audio-phonic or audio-visual processing device, comprising means for implementing the method. This device can be produced in the form, for example, of a general-purpose computer capable of executing the computer program according to the second aspect above.

Finally, a fourth and final aspect of the invention relates to an audio-phonic or audio-visual processing device such as an editing and/or mixing console making it possible to produce media (namely audio, audiovisual content, or multimedia) corresponding to or incorporating a speech signal from a speaker, in particular from a speaker to be protected, the apparatus comprising a device according to the third aspect.

Brief description of the figures

• Figure 1 : un organigramme illustrant les principales étapes du procédé selon des modes de mise en oeuvre ;
• Figure 2 : une représentation schématique très simplifiée d'un système audio-phonique dans lequel le procédé peut être implémenté ;
• Figure 3A et figure 3B : des schémas illustrant des modes de mise en oeuvre de l'altération descendante et de l'altération montante, respectivement, qui peuvent être appliquées au timbre et à la hauteur d'un segment de signal audio selon des modes de mise en oeuvre ;
• Figure 4A et figure 4B : des diagrammes de fréquence d'une séquence audio enregistrée, montrant la répartition de l'énergie en fonction de la fréquence avant et après, respectivement, la mise en oeuvre d'un procédé de masquage de la voix selon l'art antérieur ;
• Figure 5A et figure 5B : des diagrammes de fréquence de la même séquence audio qu'à la figure 4A, montrant la répartition de l'énergie en fonction de la fréquence avant et après, respectivement, la mise en oeuvre d'un procédé de masquage de la voix selon le procédé proposé ;
• Figure 6 : un organigramme détaillé illustrant les étapes du procédé selon des modes de mise en oeuvre.

The description which follows with reference to the appended drawings, given as non-limiting examples, will make it clear what the invention consists of and how it can be carried out. In the drawings, we have shown:

• Figure 1 : a flowchart illustrating the main stages of the process according to implementation modes;
• Figure 2 : a very simplified schematic representation of an audio-phonic system in which the method can be implemented;
• Figure 3A And Figure 3B : diagrams illustrating modes of implementation of the descending alteration and the ascending alteration, respectively, which can be applied to the timbre and the pitch of an audio signal segment according to modes of implementation;
• Figure 4A and Figure 4B : frequency diagrams of a recorded audio sequence, showing the distribution of energy as a function of frequency before and after, respectively, the implementation of a voice masking method according to the prior art;
• Figure 5A and Figure 5B : frequency diagrams of the same audio sequence as in the figure 4A , showing the distribution of energy as a function of frequency before and after, respectively, the implementation of a voice masking method according to the proposed method;
• Figure 6 : a detailed flowchart illustrating the steps of the process according to implementation modes.

Description of embodiment(s)

In the figures, and unless arranged otherwise, identical elements will bear the same reference signs.

The human voice is the set of sounds produced by the friction of the air of the lungs on the folds of the larynx of the human being. The pitch and resonance of the sounds made depend on the shape and size of not only their vocal cords, but also the rest of the person's body. Vocal cord size is one source of the difference between male and female voices, but it is not the only one. The trachea, mouth, pharynx, for example, define a cavity in which the sound waves emitted by the vocal cords resonate. Additionally, genetic factors cause the difference in vocal cord size within people of the same sex.

Given all these characteristics that are unique to each person, the voice of each human being is unique.

The method makes it possible to mask a speaker's voice in order to protect their identity and/or their privacy.

In what follows, we designate by original speech signal the audio signal corresponding to an acquired sequence of the undistorted speaker's voice. By masked audio signal we mean the result of the processing of the original speech signal obtained by implementing the method.

According to the modes of implementation as proposed, the protection of the identity and/or privacy of the speaker is achieved by an intentional alteration not only of the pitch but also of the timbre of the speaker's voice. This alteration is carried out by digital signal processing techniques, on the basis of processing algorithms implemented by computer.

A complex sound of fixed pitch can be analyzed into a series of elementary vibrations, called natural harmonics, whose frequency is a multiple of that of the reference frequency, or fundamental frequency. For example, if we consider a fundamental frequency having a value f , waves having frequency 2 f , 3 f , 4 f , ..., j × f , ... and so on are considered waves harmonics. The fundamental frequency (from which the frequencies j × f of the harmonics arise) characterizes the perceived pitch of a note, for example an “A”. The distribution of the intensities of the different harmonics according to their rank j , characterized by their envelope, defines the timbre. The same goes for a speech signal as for musical notes, speech being only a succession of sounds produced by the vocal apparatus of a human being.

Note that the timbre of a musical instrument or a voice designates the set of sound characteristics which allow an observer to identify by ear the sound produced, independently of the pitch and intensity of this sound. her. The timbre makes it possible, for example, to distinguish the sound of a saxophone from that of a trumpet playing the same note with the same intensity, these two instruments having their own resonances, which distinguish sounds when listening: the sound of a saxophone contains more energy on the harmonics of relatively lower frequencies which gives a relatively more “dull” sound timbre, while the timbre of the sound of a trumpet has more energy on the harmonics of relatively higher frequencies in order to give a “clearer” sound, although having the same fundamental frequency. For the voice, we designate by vocal register the set of frequencies emitted with an identical resonance, that is to say the part of the vocal range in which a singer, for example, emits sounds of respective pitches with a almost identical stamp.

The organization chart of the figure 1 schematically illustrates the main stages of the process of masking a speaker's voice. The method can be implemented in an audio-phonic system 20 as represented very schematically in the figure 2 . This system may include hardware means 201 and software means 202 allowing this implementation.

It will be noted that even if the invention concerns the masking of a speech signal, which by nature is an audio signal, this signal can belong to an audiovisual program (mixing sound and images), such as a video of the interview. of a witness wishing and/or having to remain anonymous, filmed for example using a “hidden camera” or accompanied by blurring of the image of the witness to be protected. In other words, the speech signal can correspond to all or part of the soundtrack of a video, and generally of any audio-phonic, radio, audiovisual or multimedia program.

The audio-phonic system 20 is for example audiovisual mixing equipment, used to edit video sequences in order to produce an audiovisual program from various video sequences and their respective “soundtracks”.

The hardware means 201 of the audio-phonic system 20 comprise at least one calculator, such as a microprocessor associated with random access memory (or RAM), and means for reading and recording of digital data on digital recording media (mass memory such as an internal hard disk), and data interfaces for exchanging data with external devices. To the figure 2 , we have symbolically represented an audio signal acquisition device 31 such as a microphone (or micro), as well as a data storage device 22 such as a USB key. As a variant or in addition, the system 20 can communicate in reading and/or writing with other external data carriers, in order to read thereon the data of an audio signal to be processed and/or to record the data of the audio signal thereon. audio signal after processing. Alternatively or in addition, in addition, the system 20 may include means of communication such as a modem or an Ethernet, 4G, 5G network card, etc., or even a Wi-Fi or ^Bluetooth® communication interface.

The software means 201 of the audio-phonic system 20 comprise a computer program which, when loaded into the RAM and executed by the processor of the audio-phonic system 20, is adapted to execute the steps of the method of masking the signal from a speaker.

With reference to the organization chart of the figure 1 , in step 11 the sound of the speaker's voice is captured via the microphone 31 of the system 20, either for immediate processing in the system 20, or for deferred processing.

By immediate processing we mean processing carried out during the acquisition of the audio signal, without an intermediate step of fixing this audio signal on any permanent recording medium. The data from the original audio signal then only passes through the RAM (non-permanent memory) of the system 20.

Conversely, deferred processing means processing which is carried out from a recording, made within or under the control of the audio-phonic system 20, of the speaker's speech signal acquired via the microphone 31. This recording is fixed on a mass data storage medium, for example a hard disk internal to the system 20. It can also be a peripheral hard disk, that is to say external, coupled to this system. It may also be another peripheral data storage device with permanent memory capable of storing the audio data of the speech signal permanently, such as a USB key, a memory card (Flash type or other) or an optical or magnetic recording medium (audio CD, CD-Rom, DVD, Blu-Ray disc, etc.).

The mass data storage medium can also be a data server with which the audio-phonic system 20 can communicate to download (" upload " in English) the data of the audio signal so that they are stored there, and for later download them for subsequent processing. This server can be local, that is to say part of a local network of the LAN type (from the English " Local Area Network ") to which the audio-phonic system 20 also belongs. The data server can also be a remote server, such as a data server in the Cloud which is accessible via the open Internet.

Alternatively, the speech signal corresponding to the speaker's speech sequence may have been acquired via other equipment, distinct from the audio-phonic system 20 which implements the method of masking the speaker's voice. In this case, an audio data file encoding the speaker's voice may have been recorded on a removable data medium, which can then, in step 11, be coupled to the audio-phonic system 20 for reading the audio data. This audio data file may also have been downloaded to a data server in the Cloud, to which the audio-phonic system 20 can also access in order to download the audio data of the audio signal to be processed. In all these situations, step 11 of the method then consists solely, for the audio-phonic system 20, of accessing the audio data of the speaker's speech signal.

In all cases, step 11 of the method comprises a (temporal) division of the original speech signal, into a series of successive audio segments of determined duration, which is constant from one segment to another in the series of segments. thus produced. Preferably, the division of the audio signal into a series of successive audio segments of the same determined duration is carried out by temporal windowing which is independent of the content of the audio signal, and which can be done "on the fly".

By the expression "independent of the content of the audio signal", we mean that the windowing is independent of both the frequency content, that is to say the distribution of energy in the frequency spectrum of the audio signal , and the informational or linguistic content, that is to say the semantics and/or the grammatical structure of the speech contained in this audio signal, in the language spoken by the speaker. The method is therefore very simple to implement, since no physical or linguistic analysis of the signal is necessary to generate signal segments to be processed.

In signal processing, a temporal windowing operation makes it possible to process a signal of length voluntarily limited to a duration τ , knowing that any calculation can only be done on a finite number of values. To observe or process a signal over a finite duration, we multiply it by an observation window function also called a weighting window and denoted h(t). The simplest, but not necessarily the most used or preferred, is the rectangular window (or door) of size m defined as follows: $$h\left(t\right)=\{\begin{array}{l}1,\mathit{if}t\in \left[0,\mathrm{<figure-callout\; id="0"\; label="m"\; filenames="imgf0002.png"\; state="\{\{state\}\}">m</figure-callout>}\right]\\ 0,\mathit{Otherwise}\end{array}$$

où τ désigne l'indice relatif du temps dans le segment.By multiplication (by digital calculation) of the digitized audio signal S(t) by the gate function h(t) above, then shifting, we obtain a finite series formed of a determined number N of audio signal segments s _k ( τ ), each of the same fixed duration D, and indexed by the letter k , denoted: $${\left\{s_k\left(\mathrm{\tau }\right)\right\}}_{k=1,2.3,\dots \mathrm{NOT}}$$

where τ denotes the relative index of time in the segment.

Advantageously, the duration D of an audio segment s _k ( τ ) is equal to a fraction of a second, for example between 10 milliseconds (ms) and 100 ms (in other words, D ∈ [10 ms, 100 ms ]). An audio segment then has a duration less than that of a word of the language spoken by the speaker, whatever the language in which he or she speaks. This duration is a fortiori less than the duration of a sentence or even a portion of a sentence in this language. The duration of an audio segment s _k ( τ ) is then, at most, of the order of the duration of a phoneme, that is to say the duration of the smallest speech unit ( vowel or consonant). An audio segment s _k ( τ ) therefore does not carry, in itself, any informational content with regard to spoken language, because its duration is far too short for that. This gives the masking process the advantage of simplicity, and in addition good robustness against the risk of reversion.

Note that such a decomposition of the audio signal S(t) into a series { s _k (τ)} _{k =1,2,3,... N} of segments also called elementary frames and indexed by the letter k below afterwards, obtained by windowing and shifting is classic in signal processing, because it allows the signal to be processed in successive time slices.

Step 11 also includes the formation of a series of pairs of audio segments each comprising a primate and a duplicate of an audio segment of the series of audio segments above. As will be seen in more detail later, with reference to the step diagram of the Figure 6 , these couples can more particularly be defined in the frequency domain, after Fourier transform (TF) applied to the segments s _k ( τ ) of the audio signal in the time domain. In each pair formed by a primate and a duplicate of a segment of the original speech signal of the speaker, these two elements are identical to each other, and come from the same segment considered of the original speech signal of the speaker. In what follows and in the figures of the appended drawings, the series of primates and the series of duplicates of the audio segments of the speech signal which are thus produced, undergo processing for each primate and each duplicate of the audio segment of a pair, to extract on the one hand, the envelope of the harmonics characterizing the timbre of the audio segment, and on the other hand, the signal characterizing the pitch of the audio segment. In the figure 1 , the series of timbres and the series of pitches are designated indifferently by the letters A and B, or vice versa).

For each pair of segments, the signals characterizing the pitch and the timbre extracted from the primate and the duplicate undergo parallel processing, essentially independent of each other. These treatments are illustrated by steps 12a and 13a of the left branch and by steps 12b and 13b of the right branch, respectively, of the algorithm illustrated schematically by the flowchart of the figure 1 .

Step 12a is a first ascending alteration (denoted MODa, in the following and in the drawings), applied to each element of the series A of the audio segments. This ascending alteration is not identical from one element to another in the A series. On the contrary, it evolves as a function of at least a first masking parameter. On the other hand, whatever the evolution of the first masking parameter, this first ascending alteration always has the effect of raising a determined part of the frequency content of the primacy of the audio segment to which it is applied. By this we mean that all or part of the primacy frequencies of the segment considered are shifted towards high frequencies, compared to the corresponding audio segment of the original speech signal. Applying the first alteration generates an altered timbre (here towards the top) of the audio segment.

Step 12b is a second, descending alteration (denoted MOD _B in the following and in the drawings), applied to each element of series B of audio segments. Just like the ascending alteration MOD _A applied to the elements of series A, this descending alteration MOD _B is not identical from one element to another in series B. This means that it evolves, depending on at least one second masking parameter. On the other hand, whatever the evolution of this second masking parameter, this downward alteration always has the effect of lowering a determined part of the frequency content of the element of the audio segment to which it is applied. By this we mean that all or part of the frequencies of the audio segment considered are shifted towards low frequencies, compared to the corresponding audio segment of the original speech signal. Applying the second alteration generates an altered pitch (here towards the bottom) of the audio segment.

It will be noted that it is then advantageous for each of the alterations MOD _A and MOD _B to be restricted from the point of view of the evolution of the frequency content of the elements of the audio segment to which it is applied. By this we mean that these alterations of the frequency spectrum are each only ascending or only descending, without any inflection in the direction of movement of the frequencies concerned in the spectrum considered. In fact, this prevents the audio-phonic system 20 from being able to be used itself by malicious people to whom it has been provided or made available, or who could have access to it by any other means, in order to reverse the alteration of the audio signal. Such a reversion could in fact consist of applying to the masked audio signal (which the malicious third party would have copied or intercepted in any way), alterations with judiciously chosen masking parameters to return to the original speech signal, this is i.e. to the audio signal corresponding to the natural voice of the speaker. But thanks to the modes of implementation described above, such a maneuver is not possible with the audio-phonic system 20 according to the invention itself. Indeed, no change in the values of the masking parameters of the ascending alteration MOD _A and the descending alteration MOD _B that the malicious third party could attempt can have the effect of reversing the unidirectional movements of the pitch (height) and of the timbre, respectively, of the original speech signal. In other words, the audio system 20 does not offer the possibility of reversibility of the alteration that it produces. This does not prohibit a malicious third party from attempting this fraud with other means, but at least the system used to mask the audio signal containing the natural voice of a speaker cannot be diverted from its function, in fact. “returned”, in order to remove the protection of the speaker that it provides.

The method then comprises a step 15 of combining the timbre of the audio segment, altered by the MOD _A alteration and which was obtained in step 12a, on the one hand, and the pitch of the audio segment, altered by the alteration MOD _B and which was obtained in step 12b, on the other hand, to form a single resulting altered audio segment. By combination, we mean here an operation having, from a physical point of view, the effect of recombining the respective altered spectra, that is to say of merging the respective frequency contents of the altered timbre of the audio segment and of the altered pitch. of said audio segment, possibly with averaging and/or smoothing. In signal processing, this can be obtained by multiplication (“×” symbol) or by convolution (“∗” symbol), either in the time domain or in the frequency domain after transformation of the audio signal(s). s) from the time domain into the frequency domain by a Fourier transform.

• à l'étape 13a pour les éléments de la série A, une variation d'au moins un paramètre de l'altération MOD_A, par exemple une variation de cette altération dans un intervalle de largeur paramétrable, cette variation étant notée symboliquement VAR_A dans ce qui suit et aux figures, et ;
• à l'étape 13b pour les éléments de la série B, une variation d'au moins un paramètre de l'altération MOD_B, par exemple une variation de cette altération dans un intervalle de largeur elle-même paramétrable, cette variation étant notée symboliquement VARs dans ce qui suit et aux figures,

lesdites variations des altérations étant variables d'un couple de segments à l'autre dans la série des couples de segments audios.The method further comprises, from one pair of audio segments to another in the series of pairs of audio segments:

• in step 13a for the elements of series A, a variation of at least one parameter of the alteration MOD _A , for example a variation of this alteration in a configurable width interval, this variation being symbolically denoted VAR _A in what follows and in the figures, and;
• in step 13b for the elements of series B, a variation of at least one parameter of the alteration MOD _B , for example a variation of this alteration in a width interval itself configurable, this variation being noted symbolically VARs in the following and in the figures,

said variations of the alterations being variable from one pair of segments to another in the series of pairs of audio segments.

Those skilled in the art will appreciate that, in practice, steps 12a and 12b on the one hand, and steps 13a and 13b on the other hand, can be carried out in the reverse order to that presented in the figure 2 . In other words, they are interchangeable: steps 13a and 13b can be executed after (as shown) or before steps 12a and 12b.

Preferably, steps 13a and 13b cause a local disturbance, around the instant τ , of the (spectral) characteristics of the timbre and the pitch (pitch), said disturbance varying from one segment to another in the series { s _k (τ)} _{k =1,2,3,... N} (therefore as a function of k) in a random, non-stationary manner (for example, by random walks) and independently on each of the two spectral components, namely the height or pitch and the timbre.

In an example of implementation which is however not limiting, the alteration of the height of the audio signal can thus correspond to an “oriented” variation, namely a rise or fall, of the fundamental frequency of the audio signal, which can take any of the following determined values: ± 6.25%, ± 12.5%, ± 25%, ± 50% and ± 100%. These example values correspond approximately to variations of a semitone, tone, third, fifth, or octave, respectively, in pitch (i.e. of the fundamental frequency, or “pitch”) of the original speech signal.

The sequential repetition of step 14 for the successive pairs of primates and duplicates of the audio segments generated in step 11, generates a series of altered audio segments.

The method finally comprises, in step 15, the recomposition of the masked audio signal from the series of altered audio segments obtained by the repetition of the previous steps, 12a-12b, 13a-13b and 14. This recomposition is carried out by superposition-addition, in the time domain, of the successive elements of the series of altered audio segments produced in step 14, as they are transformed.

Note that, in the resulting altered audio segment, the frequency content is doubly altered, compared to the spectrum of the considered segment of the original speech signal. This results from the accumulation of the respective effects of the MOD _A and MOD _B functions.

In embodiments, the successive changes of the first masking parameter and the second masking parameter which occur at each occurrence of steps 13a and 13b, respectively, induce random variations of said first parameter and second parameter, of a pair to the other in the series of pairs of audio segments generated in step 11.

As the MOD _A and MOD _B alterations relate to different components of the spectrum of the segment considered of the original speech signal, as in addition they use distinct masking parameters, and as finally their respective masking parameters evolve independently of one of the other randomly, the masking effect which is obtained is very difficult, if not impossible, to reverse.

Thus, the variations of the first and second masking parameters are themselves fluctuating randomly, from one pair of segments to another in the series of pairs of audio segments. In other words, the variations denoted VAR _A and VARs in steps 13a and 13b of the parameters of the modifications denoted MOD _A and MOD _B introduced in steps 12a and 12b fluctuate, as a function of time. In particular, this fluctuation occurs from a segment to another of the original speech signal. Therefore, on the figure 1 , this fluctuation is symbolized by an operation denoted VAR _A+B in step 14.

There Figure 3A and the Figure 3B illustrate a mode of implementation of the descending alteration and the ascending alteration, respectively, which can be applied to the timbre and the height (or pitch) of an audio signal segment, in step 12a and at step 12b, respectively, of the method illustrated by the flowchart of the figure 1 .

In this example, the ascending alteration MOD _A is applied to the pitch of the voice, symbolized at the Figure 3A by a tuning fork. A tuning fork is known as an object whose acoustic resonance produces a sound with a pure frequency, such as the fundamental frequency (or pitch) of a human voice. In addition, the descending alteration MOD _B is applied to the timbre of the voice, symbolized at the figure 4A by the envelope of the frequency spectrum of an audio signal. Of course, the example represented by the figures 3A And 3B is not limiting. The ascending alteration MOD _A can conversely be applied to the fundamental frequency (pitch) while the descending alteration MOD _B would be applied to all or part of the harmonic envelope (timbre).

In all cases, the two alterations MOD _A and MOD _B each produce displacements of certain frequencies (namely, in the example considered here, the pitch for one, and the envelope of the harmonics for the other) following opposite directions in the frequency spectrum (i.e. an ascending direction towards the treble for one, and a descending direction towards the bass for the other). In the protected audio signal which is obtained, these effects operating in two different directions allow good protection while preserving a certain intelligibility of the audio signal. Indeed, the “masculinizing” effect of a frequency shift towards the bass which results from the ascending alteration MOD _A is partly counterbalanced by the “feminizing” effect of a frequency shift towards the treble which results from the descending alteration MOD _A. This avoids generating a masked signal close to the voice of “Darth Vader” ^™ or close to the voice of “Mickey Mouse” ^™ .

The audio file obtained after implementing the process of figure 1 may be transmitted by electronic mail, posted online on social networks or on a website, broadcast on the airwaves, or distributed on any recording medium. The process makes it possible to mask the voice a posteriori, on a recording of the speaker's voice, as can easily be done with audio editing software. As it is offered in the form of a computer program as a plugin to be integrated into audio-phonic or audio-visual processing software, the method does not allow audio or video calls to be made with a masked voice.

Since the process is implemented on the audio-phonic or audio-visual platform with which the speaker's voice is acquired, the original voice does not circulate on any computer network, which avoids a risk of interception by a malicious third party of data corresponding to the unmasked voice.

The computer program which implements the masking method, by carrying out the calculations of the corresponding digital processing, can be included in a host software, for example the operational software of an audio-phonic processing environment, such as a audio mixing or audio-visual editing console.

• soit sur une piste distincte, ajoutée « en insert » dans le programme en cours de composition sur le système de traitement audio-phonique ou audio-visuel;
• soit directement sur le fichier audio d'origine qui a été traité, par exemple en remplacement des données du signal de parole original afin de supprimer l'enregistrement d'origine de la voix du locuteur et garantir ainsi sa protection perpétuelle.

The result obtained by implementing the method, namely the masked audio signal, can be fixed, that is to say recorded:

• either on a separate track, added “as an insert” in the program being composed on the audio-phonic or audio-visual processing system;
• or directly on the original audio file which has been processed, for example to replace the data of the original speech signal in order to delete the original recording of the speaker's voice and thus guarantee its perpetual protection.

This result is irreversible in audio, and cannot be analyzed by voice recognition. It is readable immediately, that is to say we can play the audio data file or read the corresponding audio track, in order to listen to the masked audio signal, in particular to check by ear or by any other means available technique that the original voice of the speaker is no longer recognizable.

There Figure 4A is a frequency diagram of a recorded audio sequence, showing the distribution of energy as a function of time (on the abscissa) and frequency (on the ordinate). There figure 4B is a frequency diagram of the audio sequence of the Figure 4A after the implementation of a voice masking process according to the prior art, by simple pitch shift. We sometimes speak of a “pitched” signal to designate the signal having undergone such an offset. We can clearly see by comparing these two frequency diagrams that there is a very strong analogy of the harmonics of the signal between the original signal and the pitched signal.

There figure 5A and the Figure 5B allow you to compare the frequency diagrams of the same audio sequence as in the figure 4A , by showing the distribution of energy as a function of frequency before and after, respectively, the implementation of a voice masking method according to the proposed method. This comparison shows that the harmonics of the signal have undergone significant transformations. We clearly distinguish at the Figure 5B that harmonics of the figure 5A have undergone significant modifications, thus masking the harmonics of the original signal. This masking makes it extremely difficult, if not impossible, to compare the spectrograms of the original speech signal and the masked speech signal.

Modes of implementation of the process presented above schematically and in its main stages only, will now be described in more detail with reference to the flowchart of the Figure 6 .

The implementation of the method consists of applying digital processing, here for example in the time-frequency domain which is better suited to this type of processing by calculations, following { s _k (τ)} _{k =1.2, 3,...} segments s _k ( τ ) of the digitized speech signal S(t). Such a segment is denoted s _k ( τ ) at the top of the Figure 6 . Those skilled in the art will appreciate that in practice the treatment illustrated by the step diagram in this figure is obviously applied successively to each segment s _k ( τ ) indexed by the letter k.

The segment s _k ( τ ) is the subject in step 61 of a Fourier Transform (TF), for example a short-term Fourier transform known by the acronym TFCT (or STFT, from the English " Short Term Fourier Transform ") in order to move into the time-frequency domain. Each segment s _k ( τ ) of duration τ in the time domain is thus converted to give a segment denoted S _k ( t , f ) which takes complex values in the time-frequency domain.

où : $$X_k\left(t,f\right)=\Vert S_k\left(t,f\right)\Vert$$

; et, $$Q_k\left(t,f\right)=\exp \left(i\times \mathit{Arg}{S}_k\left(t,f\right)\right),$$

où Ar g désigne l'argument d'un nombre complexe.In step 62, the segment S _k ( t , f ) is decomposed into a module term denoted X _k ( t, f) and a phase term denoted Q _k (t, f). These terms are such as: $$S_k\left(t,f\right)=X_k\left(t,f\right)\times Q_k\left(t,f\right)$$

Or : $$X_k\left(t,f\right)=\Vert S_k\left(t,f\right)\Vert$$

; And, $$Q_k\left(t,f\right)=\exp \left(i\times \mathit{Arg}{S}_k\left(t,f\right)\right),$$

where Ar g denotes the argument of a complex number.

The _term _ _ From this _term and on the other hand to estimate the envelope of the Power Spectral Density, that is to say the timbre.

où :

• A_k (t, f) correspond, pour le segment du signal d'indice k considéré, au signal caractérisant le timbre du signal audio ; et,
• B_k (t, f) correspond, pour ce segment, au signal caractérisant la hauteur (ou pitch) du signal audio.

More particularly, in step 63 we proceed to the formation of a pair of segments initially equal to each other and equal to the module term X _k ( t, f) of the segment S _k ( t, f ), and that we calls, for the purposes of this presentation, the primacy and the duplicate of the segment S _k ( t, f ) . We will also sometimes speak of a series of pairs each formed (that is to say for each value of the index k ) by this primate and this duplicate of the segment S _k ( t , f ) . Differentiated treatments applied to the primacy and the duplicate, respectively, of the segment make it possible to separate the module term X _k ( t, f) into two distinct components A _k ( t , f ) and B _k ( t , f ) such that, in the time-frequency domain, we have: $$X_k\left(t,f\right)={\mathrm{HAS}}_k\left(t,f\right)\times B_k\left(t,f\right),$$

Or :

• A _k ( t, f ) corresponds, for the segment of the signal of index k considered, to the signal characterizing the timbre of the audio signal; And,
• B _k ( t , f ) corresponds, for this segment, to the signal characterizing the height (or pitch) of the audio signal.

For example, the timbre component A _k ( t , f ) can be obtained by the cepstrum method. To this end, we apply an Inverse Fourier Transform (IFFT), and we then obtain the cepstrum, which is a dual temporal form of the logarithmic spectrum (the frequency domain spectrum becomes cepstrum in time domain). After this transformation, the fundamental frequency can be calculated from the cepstral signal by determining the index of the main peak of the cepstrum and we obtain, by windowing the cepstrum, the envelope of the spectrum which corresponds to the timbre component A _k ( t , f ) .

The height component (or pitch) B _k ( t , f ), for its part, can then be obtained by dividing the signal X _k ( t , f) point to point by the value of the timbre component A _k ( t , f ) . In other words, to obtain the height (or pitch) component B _k ( t , f ), we can “subtract” (which is carried out by a division calculation in the time-frequency space) from the term of _module _ _ _{_} _ _ _{_} _ _ characterizing the height (or pitch) or more generally what is called the fine structure of the Spectral Power Density (PSD).

• d'une part, par une fonction d'altération de l'échelle fréquentielle Γ_A (f) ou Γ_B (f), à l'étape 65a pour la composante de timbre A_k (t, f) et à l'étape 65b pour la composante de hauteur B_k (t, f), respectivement, qui sont de préférence monotones et dont l'une est ascendante alors que l'autre est descendante relativement à ses effets sur le contenu fréquentiel du segment audio original S_k (t, f) ; et,
• d'autre part, par une fonction de variation temporelle γ_A (t) ou γ_B (t) appliquées globalement à l'échelle des fréquences, à l'étape 64a pour la composante de timbre A_k (t, f) et à l'étape 64b pour la composante de hauteur, B_k (t, f) respectivement.

In steps 64a and 65a, on the one hand, and in steps 64b and 65b, on the other hand, alterations are then applied, ascending or descending, to the envelope A _k ( t , f ) of the spectrum corresponding to the timbre and to the fine structure B _k ( t , f ) of the spectrum corresponding to the height, according to a preferably monotonous transformation along the frequency axis, these alterations being distinct from each other as to their implementation modalities implemented, and being moreover each variable randomly, from one audio signal segment to another. These alterations make it possible to modify respectively the timbre and the pitch of the signal independently, and variable over time (non-stationary), more particularly from one audio signal segment to another. that is to say according to the index k . For each of the timbre and the pitch, this result is obtained globally by the multiplication in the time-frequency domain of the component A _k ( t , f ) or B _k ( t , f ), respectively, of the power spectral density _Xk ( t , f ) :

• on the one hand, by an alteration function of the frequency scale Γ _A ( f ) or Γ _B ( f ), in step 65a for the timbre component A _k ( t , f ) and in step 65b for the pitch component B _k ( t , f ), respectively, which are preferably monotonic and one of which is ascending while the other is descending relative to its effects on the frequency content of the original audio segment S _k ( t , f ); And,
• on the other hand, by a temporal variation function γ _A ( t ) or γ _B ( t ) applied globally to the frequency scale, in step 64a for the timbre component A _k ( t , f ) and at step 64b for the height component, B _k ( t , f ) respectively.

The order in which these operations are carried out in the time-frequency domain is immaterial. In the implementation shown in Figure 6 , we first carry out the multiplications of the components A _k ( t, f) and B _k ( t , f) by the time variation functions γ _A ( t ) and γ _B ( t ), respectively, and we then carry out the multiplications respective results of these first multiplications by the frequency alteration functions Γ _A ( f ) and Γ _B ( f ), in step 65a and in step 65b, respectively. But these two groups of multiplications could just as easily be carried out in reverse order. In other words, steps 65a and 65b could be carried out before steps 64a and 64b, respectively.

As will be understood, and as it is shown on the left of the blocks illustrating steps 64a, 64b, 65a and 65b in the Figure 6 , in these modes of implementation the frequency alteration functions Γ _A ( f ) and Γ _B ( f ) correspond to the alterations MOD _A and MOD _B , respectively, which were presented above with reference to the figure 1 . Likewise, the time variation functions γ _A ( t ) and γ _B ( t ), correspond to the variations VAR _A and VARs, respectively, which were presented above with reference to the figure 1 . To avoid any ambiguity, it should be noted that, from the point of view of the frequency spectrum of the original audio segment S _k ( t , f), which varies in the time by the effect of the temporal variation functions γ _A ( t ) and γ _B ( t ), this is the overall effect on this spectrum and more particularly on the timbre and on the pitch, respectively, of the combination of functions of frequency alteration Γ _A ( f ) and Γ _B ( f ) and of the temporal variation functions γ _A ( t ) and γ _B ( t ), respectively, that is to say of the accumulation (or of the addition) of their respective effects. These respective effects of the combination of the frequency alteration functions Γ _A ( f ) and Γ _B ( f ) and the time variation functions γ _A ( t ) and γ _B ( t ) on the frequency spectrum of the original audio segment is more particularly linked to the frequency alteration functions Γ _A ( f ) and Γ _B ( f ), respectively, the temporal variation functions γ _A ( t ) and γ _B ( t ) only having the effect of making them vary, preferably randomly, in order to reinforce the robustness of the masking against attempts at reversion due to malicious intent.

In the example shown in Figure 6 , step 64a comprises the application to the signal A _k ( t , f) which corresponds to the timbre component, on the frequency scale f , of the temporal variation function γ _A ( t ), to generate a signal intermediate, denoted A' _k ( t, f ), of the timbre component A _k ( t , f ) . This operation can be written as a multiplication in the time-frequency domain, as follows: $$\mathrm{HAS}{\prime }_k\left(t,f\right)={\mathrm{HAS}}_k\left(t,f\times {\gamma }_{\mathrm{HAS}}\left(t\right)\right)$$

The function _γA ( t ) is a linear function. Preferably, and as has already been mentioned above, it fluctuates over time in a random manner, varying from one original audio signal segment to another in the series of segments S _k ( t , f ) which are processed in sequence. In other words, it changes as a function of the value of the index k, according to a random process whose refreshing is governed by a parameter θ , so that the alteration of the timbre is not stationary.

La fonction γ_B (t) est une fonction linéaire. De préférence, et ainsi qu'il a déjà été mentionné plus haut, elle fluctue au cours du temps de manière aléatoire, en variant d'un segment de signal audio original à l'autre dans la série des segments S _k (t, f) qui sont traités en séquence. Dit autrement, elle change en fonction de la valeur de l'indice k selon un processus aléatoire dont le rafraîchissement est gouverné par un paramètre θ en sorte que l'altération de la hauteur n'est pas stationnaire.In the same way, step 64b includes the application to the signal B _k ( t , f ), which corresponds to the height component (or pitch=, on the frequency scale f , of the temporal variation function γ _B ( t ), to generate an intermediate signal, denoted B' _k ( t, f ). This operation can be written as a multiplication in the time-frequency domain, as follows: $$B{\prime }_k\left(t,f\right)=B_k\left(t,f\times {\gamma }_B\left(t\right)\right)$$

The function _γB ( t ) is a linear function. Preferably, and as has already been mentioned above, it fluctuates over time in a random manner, varying from one original audio signal segment to another in the series of segments S _k ( t , f ) which are processed in sequence. In other words, it changes according to the value of the index k according to a random process whose refreshing is governed by a parameter θ so that the alteration of the height is not stationary.

The fluctuations, as a function of time, of the temporal variation function γ _A ( t ) applied to the timbre component and/or of the temporal variation function γ _B ( t ) applied to the pitch component (height), and all the more so when one and/or the other of these fluctuations is random, makes it possible to reinforce the irreversibility of the voice masking process.

For example, the temporal variation function γ _A ( t ) can vary according to a random walk within a determined amplitude range [ δ _A ^min , δ _A ^max ] and with a temporal refresh rate corresponding to the parameter θ mentioned above, where δ _A ^min , δ _A ^max and θ are first masking parameters, associated with the temporal variation function γ _A ( t ) .

In the same way, the temporal variation function γ _B ( t ) can for example vary according to a random walk within an amplitude range [ δ _B ^min , δ _B ^max ] and with a temporal refresh rate corresponding to the aforementioned parameter θ , where δ _B ^min , δ _B ^max and θ are second parameters, associated with the temporal variation function γ _B ( t ) . The fluctuations of the two temporal variation functions γ _A ( t ) and γ _B ( t ) are preferably independent of each other, in order to reinforce the irreversibility of the alterations. In other words, the temporal variation functions γ _A ( t ) and γ _B ( t ) are uncorrelated.

We will appreciate that the parameter θ is the parameter of the fluctuation denoted VAR _A+B at the figure 1 . This parameter defines, for example, the number of random variations per second of alterations in the spectrum of an audio segment. For example, if θ were equal to zero, the variations VAR _A and VARs are stationary, so that the results of the alterations MOD _A and MOD _B would be fixed, which is not the case in practice. In an example θ has a value between 1 and 10. This value being homogeneous at a frequency, we can say that θ is between 1 and 10 Hz. This value is lower than the frequency of the temporal division of the original speech signal into audio segments (by windowing) , which is more of the order of 100 Hz.

Then, in steps 65a and 65b, frequency alteration functions Γ _A ( f ) and Γ _B ( f ) are applied, respectively, to the timbre component A _k (t, f) and to the pitch component B _k ( t , f ), respectively, to generate a timbre component of the masked audio segment, denoted A" _k ( t , f), and a pitch component of the masked audio segment, denoted B" _k ( t , f ), respectively. These frequency alteration functions Γ _A ( f ) and Γ _B ( f ) correspond to the alterations noted MOD _A and MOD _B on the figure 1 .

$$B{"}_k\left(t,f\right)=B{\prime }_k\left(t,{\Gamma }_B\left(f\right)\right)$$

These operations can each be written as a multiplication in the time-frequency domain, as follows: $$\mathrm{HAS}{"}_k\left(t,f\right)=\mathrm{HAS}{\prime }_k\left(t,{\Gamma }_{\mathrm{HAS}}\left(f\right)\right)$$

$$B{"}_k\left(t,f\right)=B{\prime }_k\left(t,{\Gamma }_B\left(f\right)\right)$$

et/ou, respectivement, $${\Gamma }_B\left(f\right)=f\times {\Gamma }_B$$

The function Γ _A ( f ) and the function Γ _B ( f ) can be linear or non-linear deformation functions of the frequency axis. In the case where one and/or the other are linear, it comes: $${\Gamma }_{\mathrm{HAS}}\left(f\right)=f\times {\Gamma }_{\mathrm{HAS}}$$

and/or, respectively, $${\Gamma }_B\left(f\right)=f\times {\Gamma }_B$$

Preferably, the alteration functions Γ _A ( f ) and Γ _B ( f ) are monotonic, that is to say that the deformation that they introduce on the frequency axis is either ascending with the effect of raising a determined part of the frequency content of the audio segment s _k ( τ ), or descending with the effect of lowering a determined part of the frequency content of the audio segment s _k (τ). Furthermore, they are constrained in the opposite direction, in the sense that, if one is ascending monotonous, the other is descending monotonous, and vice versa. This prevents the software which implements the masking process from being used itself to attempt a reversion of the speaker's voice masking process, as has already been explained above with reference to steps 12a and 12b of the figure 1 .

Furthermore, the fact that among the alteration functions Γ _A ( f ) and Γ _B ( f ) one is an ascending alteration function, while the other is a descending alteration function makes it possible to preserve the 'intelligibility of the voice after masking, since the shift(s) of frequency(ies) towards the treble, on the one hand, and the shift(s) of frequency(ies) towards the bass that they produce, on the other hand, partly compensate each other by avoiding excessive distortion of the voice, which would otherwise be predominant in the masked audio signal.

One of the advantages of the method comes from the implementations in which these modifications MOD _A and MOD _B are varied for the successive indices k according to two uncorrelated random sequences (one for the timbre and the other for the pitch or pitch), so as to permanently modify these two characteristics of the voice in an independent, unpredictable and non-stationary manner. Unlike methods where the modification would be constant, this makes it impossible to reverse the process once the frequency variations are made. The protection is all the stronger as the random variations VAR _A and VARs are important.

The following two steps make it possible to keep the temporality of the original by resynthesizing the audio signal masked by the index k .

Thus, step 67 includes the reconstruction of each modified audio segment, denoted X " _k ( t , f ), in the time-frequency domain, by recombination of the new envelope A " _k ( t, f ) and the new fine structure of the frequency spectrum B " _k ( t , f ) of the audio segment considered. The term "new" used here in reference to the envelope and the fine structure means that it is the envelope and the fine structure after masking, that is to say after application of the frequency alteration functions Γ _A ( f ) and Γ _B ( f ) correspond to the alterations MOD _A and MOD _B , respectively, and the temporal variation functions γ _A ( t ) and γ _B ( t ), respectively. This reconstruction can be obtained by multiplying in the time-frequency domain the new timbre component A " _k ( t , f ) by the new pitch component B " _k ( t , f ) of the frequency spectral density (FSD) of the masked audio segment, as follows: $$X{"}_k\left(t,f\right)=\mathrm{HAS}{"}_k\left(t,f\right)\times B{"}_k\left(t,f\right)$$

Step 68 includes the recomposition of each masked audio segment denoted S" _k ( t , f ), in the time-frequency domain. This recomposition can be obtained by multiplying in the time-frequency domain the module component X" _k ( t , f) by the corrected phase component Q" _k ( t , f) of the masked audio segment S" _k ( t , f), as follows: $$S"\left(t,f\right)=X"\left(t,f\right)\times Q"\left(t,f\right)$$

The corrected phase component Q " _k ( t , f ) of the masked audio segment S " _k ( t , f ) is obtained, in the example shown in Figure 6 , in step 66 from the phase term Q _k ( t , f ) of the audio segment considered S _k ( t , f ), which phase term was generated in step 62. Step 66 has for function of providing a correction of the phase term Q _k ( t , f) of the audio segment S _k ( t , f) as a function of the random variations γ _B ( t ) and the alteration function Γ _B ( f ) which were applied to the pitch term B(t, f). This ensures the temporal continuity of the phase Φ " _k ( t , f ), of the masked audio segment S " _k ( t , f ), that is to say the continuity of the phase Φ " _k ( t , f ) of this segment with Φ " _k ( t - 1, f ), where Φ " _k ( t , f) corresponds to Arg S " _k ( t , f).

It will be noted that such a phase correction is known per se and is generally implemented in any signal transformation processing as soon as the power spectral density of a signal is modified. In the implementation modes proposed here, it is generated in step 66 only as a function of the modifications made to the pitch component B " _k ( t , f ) of the power spectral density of the audio segment masked S " _k ( t , f ) with respect to the pitch component _Bk ( t , f ) of the power spectral density of the original audio segment _Sk ( t , f ) . Indeed, for the most part, it is the modifications made to the height (pitch) which call for a phase adjustment of the frequency components of the spectrum. However, those skilled in the art will appreciate that the phase adjustment of step 66 could also take into account the modifications made to the timbre component A " _k ( t , f ) of the frequency spectral density of the masked audio segment S " _k ( t , f) with respect to the timbre component A _k ( t , f ) of the frequency spectral density of the original audio segment S _k ( t , f ) . This is not shown on the organization chart of the Figure 6 in order not to overload it, which would harm its readability, but the person skilled in the art understands, on the basis of his usual knowledge and in view of the indications provided here, the way in which this can be implemented in practice.

Once the masked audio segment S " _k ( t , f) has been obtained by calculations in the time-frequency domain as explained in the above, all that remains is to bring it back to the time domain, which is achieved in step 69. This step consists of generating the masked signal s " _k (τ) in the time domain, from the signal S " _k ( t , f ) in the time-frequency domain. For example, this can be obtained by an OLA method (from the English “OverLap-and-Add ”) on the successive inverse Fourier Transforms of s" _k (τ) . The OLA method, also called the superposition and addition method, is based on the linearity property of linear convolution, the principle of this method consisting of decomposing the linear convolution product into a sum of linear convolution products. Of course, other methods can be considered by those skilled in the art to carry out this inverse Fourier transform, in order to generate s" _k (τ) in the time domain from S " _k ( t , f ) in the time domain. time-frequency domain.

The method which has been presented in the preceding description can be implemented by a computer program, for example as a plugin which can be integrated into audio-phonic or audio-visual processing software.

To the Figure 6 , the reference 60 collectively designates the parameters of masking the voice of a speaker, namely δ _A ^min , δ _A ^max , δ _B ^min , δ _B ^max , θ , Γ _A and Γ _B which can be adjusted by a user, via a man-machine interface adapted from the device on which the speaker's voice masking software is executed.

Claims (7)

Method of masking a speaker's voice to protect their identity and/or privacy by intentionally altering the pitch and timbre of the voice, comprising: - a division of an audio signal corresponding to an original recording of the speaker's voice into a series of successive audio segments of determined constant duration, and the formation of a series of pairs of audio segments each comprising a primate and a duplicate of 'an audio segment of said series of audio segments; as well as, for each pair of audio segments: - processing the primacy of the audio segment and processing the duplicate of the audio segment to extract, on the one hand, a signal characterizing the pitch of the audio segment, and on the other hand, a signal characterizing the timbre of the audio segment; - a first alteration (12a; 65a), applied to the signal characterizing the timbre extracted from the audio segment and having the effect of altering all or part of the envelope of the harmonics of said audio segment, so as to generate an altered timbre of the audio segment ; - a second alteration (12b; 65b), applied to the signal characterizing the height of the audio segment, and having the effect of altering the value of the fundamental frequency, so as to generate an altered height of the audio segment; one of the alterations among the first alteration (12a; 65a) and the second alteration (12b; 65b) being an ascending alteration while the other alteration is a descending alteration,
as well as, - a combination (15; 67) of the altered timbre of the audio segment and the altered pitch of the audio segment, to form a resulting altered audio segment, the method further comprising, from one pair of audio segments to another in the series of pairs of audio segments: - a variation (13a; 64a) of the first alteration and; - a variation (13b; 64b) of the second alteration, said variations of said first and second alterations being fluctuating randomly from one pair of segments to another in the series of pairs of audio segments,
and the method further comprising: - the recomposition (14) of a masked audio signal from the series of altered audio segments. Method according to claim 1, in which the division of the audio signal into a series of successive audio segments of determined duration is carried out by temporal windowing independent of the content of the audio signal. Method according to claim 1 or 2, in which the division of the audio signal is configured so that the duration of an audio segment is equal to a fraction of a second, so that successive changes of the first parameter and the second parameter occur several times. times per second. Method according to any one of the preceding claims, in which the first alteration corresponds to a variation in the fundamental frequency of the audio signal of any of the following values: ± 6.25%, ± 12.5%, ± 25%, ± 50 % and ± 100%. Computer program comprising instructions which, when the computer program is loaded into the memory of a computer and is executed by a processor of said computer, cause the computer to implement all the steps of the method according to the any of claims 1 to 4. Audio-phonic or audio-visual processing device, comprising means for implementing all the steps of the method according to any one of claims 1 to 4. Audio-phonic or audio-visual processing device such as an editing and/or mixing console making it possible to produce audio, audiovisual, or multimedia content corresponding to or incorporating a speech signal from a speaker, in particular from a speaker to be protected, the apparatus comprising a device according to claim 6.

Images