EP1460614A1 - Audio device (mobile telephone) for mixing a digital speech signal and a digital music signal - Google Patents

Abstract

The device has an analog to digital converter (8) for converting an analog speech signal into a digital speech signal. A storage unit (10) stores a set of coded data representing a musical score comprising a set of notes. A synthesizer (3) extracts the set of the coded data in a digital musical signal. A mixing unit (4) mixes a portion of the digital speech signal and digital audio signal, to produce a digital signal. An independent claim is also included for a telecommunication terminal.

Description

The present invention relates to an audio device for modifying the voice of the user of the audio device and also relates to a telecommunication termihanl for modifying the voice transmitted during a telephone communication.

Even if speech transmission remains the essential element of mobile telephony, the fact remains that manufacturers seek to differentiate their products by offering new services that are attractive and entertaining for the consumer. Examples are games, services related to voice recognition or the multiplication of ringing tones.

These new services often involve an additional cost on the phone linked to the addition of software or hardware elements.

The present invention aims to provide an audio device offering a service for modifying the voice transmitted by the user of the terminal, in particular during a telephone call, this service having an attractive and distracting nature and being implemented in a simple and economic.

• des moyens d'entrée par l'utilisateur dudit dispositif audio d'un signal de parole analogique,
• un convertisseur pour convertir ledit signal de parole analogique en un signal numérique de parole, ledit signal numérique de parole comportant au moins une fréquence fondamentale,
• des moyens pour mémoriser un ensemble de données codées représentant une partition musicale, ladite partition musicale comportant un ensemble de notes, chaque note étant définie par une fréquence fondamentale, une durée et un instrument qui joue ladite note,
• des moyens pour extraire dudit ensemble de données codées un signal numérique de musique,

   caractérisé en ce que ledit dispositif audio comporte des moyens pour mélanger une première portion dudit signal numérique de parole et une première portion dudit signal numérique de musique afin de produire un signal numérique, dit chanté.The present invention therefore proposes an audio device comprising:

• means of input by the user of said audio device of an analog speech signal,
• a converter for converting said analog speech signal into a digital speech signal, said digital speech signal comprising at least one fundamental frequency,
• means for memorizing a set of coded data representing a musical score, said musical score comprising a set of notes, each note being defined by a fundamental frequency, a duration and an instrument which plays said note,
• means for extracting from said coded data set a digital music signal,

characterized in that said audio device includes means for mixing a first portion of said digital speech signal and a first portion of said digital music signal to produce a so-called sung digital signal.

Thanks to the invention, the voice can follow the musical score

Advantageously, said audio device comprises a processor for processing digital signals DSP comprising said means for mixing said first portions of the digital speech and music signals.

Advantageously, said means for mixing said first portions of said digital speech and music signals include means for replacing the fundamental frequency of said speech signal by the fundamental frequency associated with a note of said music signal.

Advantageously, the replacement of the fundamental frequency of said speech signal by the fundamental frequency associated with a note of said music signal is carried out for a duration substantially equal to the duration of said note.

Advantageously, said audio device comprises means for adding to said digital sung signal a second portion of said digital speech signal.

Advantageously, said audio device comprises means for adding to said digital signal sung a second portion of said digital music signal.

Advantageously, said means for mixing said first portions of said digital speech and music signals include means for replacing at least one harmonic frequency of the fundamental frequency of said speech signal with a frequency harmonic of the fundamental frequency associated with a note of said music signal.

Advantageously, said audio device comprises means for discriminating a consonant from a vowel in said digital speech signal, said discrimination means activating said means for mixing said first portions of digital speech and music signals during the detection of said vowel.

Thus, the mixing of speech and music signals will take place after a consonant, therefore on a vowel. This detection can be carried out using means for detecting envelopes by sliding windows and spectral analysis.

Advantageously, said audio device comprises a voice activity detector controlling said means for mixing said first portions of the digital speech and music signals.

Thus, it can be decided to modify the fundamental frequency of the voice only after a reduction in the amplitude of said voice signal.

Advantageously, said audio device comprises a vocoder, said vocoder performing coding of said sung signal.

The present invention also provides a telecommunication terminal according to one of the preceding characteristics.

The implementation of this service on a telecommunication terminal is done in a simple and economical way using for example the DSP processor (Digital Signal Processor) of the phone.

In addition, the digital speech and music signals can be mixed in real time so that the voice is modified and then directly transmitted during a telephone call.

Advantageously, said audio device comprises means for transmitting said sung digital signal to another terminal in real time.

Other characteristics and advantages of the present invention will appear in the following description of an embodiment of the invention given by way of illustration and in no way limitative.

• La figure 1 représente schématiquement un terminal de télécommunication selon l'invention.

In the following figure:

• FIG. 1 schematically represents a telecommunications terminal according to the invention.

FIG. 1 represents a telecommunications terminal 1 according to the invention such as a mobile telephone.

• un processeur 2 de traitement de signaux DSP (« Digital Signal Processor »),
• un microphone 11,
• un haut-parleur 12,
• un convertisseur analogique-numérique 8,
• un convertisseur numérique-analogique 9,
• un élément 10 de stockage de partitions musicales définies dans un format de codage prédéterminé.

Terminal 1 includes:

• a DSP signal processing processor (“Digital Signal Processor”),
• a microphone 11,
• a speaker 12,
• an analog-digital converter 8,
• a digital-analog converter 9,
• an element 10 for storing musical scores defined in a predetermined coding format.

Musical scores can have a MIDI music coding format, Yamaha® SMAF, polyphonic EMR R5, IrDA iMelody from lrMC (Infrared Mobile Communications) or another format for vector description of music.

Each note in the musical score is characterized by its pitch, ie its fundamental frequency, and its timbre, ie the harmonics of the fundamental frequency.

The coded score includes a set of couples (note, duration). The notes are interpreted in duration and frequency, with each corresponding note a start date, an end date and several frequencies (fundamental frequency and harmonic frequencies).

The converters 8 and 9 belong, for example, to the same CODEC 13 (Encoder Decoder).

• un synthétiseur 3,
• des moyens 4 de mélange de signaux,
• des moyens 5 additionneurs de signaux,
• un vocodeur 6.

Processor 2 includes:

• a synthesizer 3,
• means 4 for mixing signals,
• means 5 signal adders,
• a vocoder 6.

The vocoder 6 is for example an adaptive multi rate AMR (Adaptive Multi Rate) vocoder for performing source coding of type 3 GPP TS 26.071 AM.

The sound of the voice is picked up by the microphone 11. The sound pressure is transformed into an analog electrical signal on a frequency band [300-3400 Hz]. This analog signal is divided into contiguous intervals of 20 ms duration. Each interval is digitized by the analog-to-digital converter 8.

A digital speech signal S1 is thus obtained in the form of frames of 20 ms.

Likewise, the synthesizer 3 makes it possible to extract a digital signal S2 of music in the form of frames of 20 ms corresponding to a partition stored in the storage element 10.

A proportion X% of the signal S1 and a proportion Y% of the signal S2 are processed by the means 4 signal mixers.

The means 4 mixers will thus replace the fundamental frequency and the harmonics of the voice signal by the fundamental frequency and the harmonics of each of the notes of the music signal during the duration of the note. This substitution is done in real time with the arrival of the sampled voice so that the voice follows the different frequencies associated with the notes of the score.

Speech is broken down using a digital filter into noises (consonants) and successive sinusoidal signals (vowels), detected as such by their waveforms; at the output of this filter, a proportion Y% of a musical sinusoidal signal deduced from the signal S2 is substituted for a proportion X% of a sinusoidal speech signal.

There is thus obtained at the output of the mixing means 4 a sung digital signal S3.

In order to maintain the intelligibility of the voice, a proportion (100-X)% of the original digital voice signal S1 is preserved and added to the signal S3 by the signal adding means 5.

Likewise, a proportion (100-Y)% of the original digital music signal S2 can be added to S3 via the adding means 5.

The mixing means 4 and adders 5 are software means integrated into processor 2.

The mixed and added signal S4 at the output of the adding means 5 is then coded by the vocoder 6 and then transmitted to an interlocutor. There is thus a real-time transmission of the signal S1 modified to follow the partition.

The coded signal can also be stored in a file having a format of the AMR IETF type. This file can then be sent to another terminal which can be for example a mobile terminal or a personal computer.

The signal S4 can also be transmitted to the digital to analog converter 8 and then transmitted to the loudspeaker 9.

Other functions not shown can be added to the processor.

It may indeed be useful not to replace the fundamental frequency and the harmonics of the voice signal by the fundamental frequency and the harmonics of a note of the music signal when the voice is on a consonant corresponding to a "glotted" sound. . Therefore, the terminal can include envelope detection means by sliding windows to detect a consonant of the digital speech signal. The mixing means are then activated only at the end of this consonant.

These detection means use a TFR (Fast Fourier Transform) spectral analysis function which behaves like a filter bank and which makes it possible to detect the presence of a power peak among the frequencies making up the line spectrum, said peak of power corresponding to the fundamental frequency of a vowel; or to detect the absence of a power peak, and therefore if there is however a signal, the presence of noise corresponding to a consonant.

In addition, the vocoder 6 of the terminal includes a voice activity detector VAD (Voice Activity Detector) making it possible to interrupt the radio transmission in the absence of a voice signal. The terminal according to the invention can advantageously use such a detector to control the mixing means. Thus, when the amplitude of the voice signal tends towards zero, the VAD detector can force the mixing means to move to the next note in the score. The VAD works in all or nothing mode, so during a silence of the voice signal long enough, a command can be sent to the mixer 4 so that we can either continue to follow the partition by providing only part of the digital music signal ((100-Y)% of signal S2 in Figure 2) on the sung digital signal, or decide to introduce silence on the sung digital signal and resume monitoring of the partition when the vocal activity resumes.

Of course, the invention is not limited to the embodiment which has just been described.

In particular, the AMR vocoder described can be replaced by any type of vocoder using source coding such as a vocoder performing RPE-LTP coding in accordance with the GSM 06.10 standard or the ETS 300 726 GSM EFR (Enhanced Full Rate).

Claims (12)

Audio device comprising: - means (11) for input by the user of said audio device (1) of an analog speech signal, a converter (8) for converting said analog speech signal into a digital speech signal (S1), said digital speech signal comprising at least one fundamental frequency, - means (10) for storing a set of coded data representing a musical score, said musical score comprising a set of notes, each note being defined by a fundamental frequency, a duration and an instrument which plays said note, - means (3) for extracting from said set of coded data a digital music signal (S2), characterized in that said audio device (1) includes means (4) for mixing a first portion of said digital speech signal (S1) and a first portion of said digital music signal (S2) to produce a digital signal (S3) , says sung. Audio device according to one of the preceding claims, characterized in that said audio device comprises a processor (2) for processing DSP digital signals comprising said means (4) for mixing said first portions of the digital speech and music signals (S1, S2). Audio device according to one of the preceding claims, characterized in that said means (4) for mixing said first portions of said digital speech and music signals (S1, S2) include means for replacing the fundamental frequency of said speech signal by the fundamental frequency associated with a note of said music signal. Audio device according to the preceding claim, characterized in that the replacement of the fundamental frequency of said speech signal by the fundamental frequency associated with a note of said music signal is produced for a duration substantially equal to the duration of said note. Audio device according to one of the preceding claims, characterized in that the said audio device comprises means (5) for adding to the said sung digital signal (S3) a second portion of the said digital speech signal. Audio device according to one of the preceding claims, characterized in that the said audio device comprises means (5) for adding to the said sung digital signal a second portion of the said digital music signal. Audio device according to one of the preceding claims, characterized in that said means (4) for mixing said first portions of said digital speech and music signals (S1, S2) include means for replacing at least one harmonic frequency of the fundamental frequency of said speech signal by a harmonic frequency of the fundamental frequency associated with a note of said music signal. Audio device according to one of the preceding claims, characterized in that said audio device comprises means for discriminating a consonant from a vowel in said digital speech signal, said discrimination means activating said means for mixing said first portions of the digital speech signals. speech and music during the detection of said vowel. Audio device according to one of the preceding claims, characterized in that said audio device comprises a voice activity detector controlling said means for mixing said first portions of the digital speech and music signals. Audio device according to one of the preceding claims, characterized in that said audio device comprises a vocoder (6), said vocoder performing coding of said sung signal. Telecommunication terminal (1) characterized in that it comprises an audio device according to one of the preceding claims. Telecommunication terminal (1) according to the preceding claim characterized in that said terminal (1) comprises means (2) for transmitting said sung digital signal (S3) in real time to another terminal.

Images