FR2695750A1 - Speech signal treatment device for hard of hearing - has speech analyser investigating types of sound-noise, and adjusts signal treatment according to speech type - Google Patents

Abstract

The signal (10) is applied to three speech analysers, looking at minimum, mean and maximum levels (11), the time structure (12) and the number of continuous cells of information (13). The outputs are applied to a detector (14) where different types of speech consonants are identified e.g. "continuous", "friction", "exploding", and "random" where the terms apply to the variation of the speech signal with time. After application to a time segmenter (15) and events selector (16), the signal (26) is applied to a speech modifier (21a-21c). The modifier is adapted to the type of speech identified, optimising the speech output. ADVANTAGE - Allows hard of hearing to receive sound signals corrected for their impairment, and improving speech recognition.

Description

DEVICE FOR PROCESSING A SOUND SIGNAL AND
APPARATUS COMPRISING SUCH A DEVICE
The subject of the present invention is a device for processing a sound signal, intended for use in the perception of speech in noise, speech therapy, learning of foreign languages, radio systems, telecommunications, cochlear implants, processing of the speech signal. divers, amplifiers (music, song, movie sounds ...), as well as in particular for the treatment of deafness.

 Deafness is usually thought of as a weakening of the feeling of intensity. This alteration is measured, in the deaf subject, by the minimum intensity in decibels which causes, on each ear, an auditory sensation, and this for different frequencies of the auditory field: the results are represented by the tonal audiogram. If the curve representative of the loss in decibels as a function of the frequencies is, in the healthy subject, a flat curve, that of the deaf subject can show points more or less distant from zero decibel according to the frequency: for example a deaf subject will be able to hear certain frequencies relatively well and weakly other frequencies.

 Hearing aids currently on the market correct the loss of sensation of intensity by amplifying the signal with response curves adapted to the shape of the tonal audiogram.

 Deafness, when caused by damage to the sensorineural auditory system, does not only cause a weakening of the feeling of intensity; it also induces two types of distortion of the auditory sensation: temporal distortions and frequency distortions, with interactions between temporal distortion, frequency distortion and intensity distortion.

 This implies that, even if a deaf subject hears speech with a suitably adapted traditional hearing aid, he will continue to hear speech in an insufficiently clear manner, in particular in noisy surroundings, and will not understand correctly the message addressed to him. .

 For speech to be correctly perceived, it is necessary that the human auditory system is able to discern sufficient useful spectral components. These useful spectral components of speech are called relevant indices. The relevant indices, according to their time, frequency and energy value, are characteristic of such a vowel, such a consonant or such a syllable. The vowels, the consonants, the syllables are pronounced one after the other in time. A deaf subject will have difficulty understanding speech because he is unable to correctly discern enough clues.

 The device for processing a sound signal, according to the present invention, is an electronic signal processing system, in real or quasi-real time, allowing electronically to detect the relevant cues useful for speech recognition and to modify them. , depending on their type and nature, in particular with respect to time: lengthening or shortening of the detected index, delay in its appearance or disappearance over time.

 Thus, thanks to the processing device of the invention, the temporal modification, in particular, of each relevant index will be adjusted according to the deafness to obtain a clearer perception of each of them; in addition, it is also possible to shorten silences between words or sentences to compensate for the lengthening chosen for certain relevant indices.

 The device for processing a sound signal, according to the invention, comprises an event detector, capable of recognizing the different sound phenomena and comprising an analyzer of the temporal structure of each event to distinguish the sequences where said event is continuous from those where it is of the explosion type, and an event modifier comprising a modification unit for modifying said temporal structure.

 Advantageously, the information delivered by said analyzer is sent to an operator detector, which determines the type of event in progress.

 Preferably, the type of event in progress is chosen from the group formed by a vowel, a consonant consonant, a friction consonant, an explosion consonant, a transition, and a speech silence, during which noise will be present, as well as all its sound carried by a signal.

 Two consecutive events are separated from each other by a time segmenter; the event modifier includes a specific processing unit provided for each of the event types; an event selector directs the event to its specific processing unit; the orientation of the event by the event selector is controlled by the operator detector; the specific processing unit includes six specialized processing circuits for processing vowels, formants, friction and explosion consonants, transitions and silences of speech or noise.

 Preferably, the event detector also comprises an analyzer of the energy of the signal of each event, said event modifier further comprising a modification unit for modifying the energy of at least one of the events .

 Advantageously, the event detector also comprises a frequency structure analyzer for distinguishing the sequences where said event has a harmonic structure, composed of formants, from those where it has a random structure, said event modifier further comprising a modification unit for modifying the frequency structure of at least one of the events.

 Of course, the measurement of temporal and frequency auditory distortions makes it possible to orient the adjustments of the treatment device, and thus to adapt it to a given deafness. The processing device according to the invention performs a different processing, on the one hand according to the type of component detected in speech, to allow the best possible perception of each relevant clue by a given deaf ear, the difficulties of perception being depending on the type of relevant index, or, on the other hand, depending on whether there is no speech, therefore presence of noise.

 It will be understood that, when the device for processing a sound signal, according to the invention, includes an event detector, capable of recognizing the different sound phenomena and comprising an analyzer of the energy of the signal of each event, a temporal structure analyzer to distinguish the sequences where said event is continuous from those where it is of the explosion type, and a frequency structure analyzer to distinguish sequences where said event has a harmonic structure, composed of formants, from those where it has a random structure, said processing device further comprising an event modifier capable of modifying the energy, the time structure and the frequency structure of at least one of the events, the best possible perception will be obtained, for a given deaf ear.

 The event detector intervenes in the distinction of the sequences characterized by the importance and the variability of their energy, as well as in the distinction of the noise.

 Thus, advantageously, the event modifier comprises three modification units, to modify the energy, the time structure, and the frequency structure of at least one of the events.

 The time structure modification unit is connected to a time offset incrementor which records the lengthening or shortening time resulting from the modification of the time structure; the incrementer also reduces the silence greater than a given value of speech or noise.

 Preferably, the events processed are reclassified chronologically in a time sequence adder whose output constitutes the output signal from the processing device.

 The invention also relates to an apparatus, comprising means for receiving a sound signal, means for processing said signal and means for restoring the modified sound signal, said means for processing said signal comprising the processing device according to the invention. 'invention.

 To better understand the object of the invention, we will now describe, by way of purely illustrative and nonlimiting example, diagrams of embodiment shown in the accompanying drawings.

On these drawings
- Figure 1 shows the functional block diagram of the processing device according to the invention;
- Figure 2 shows the part of the event modifier in Figure 1 that processes a vowel;
- Figure 3 shows the part of the event modifier in Figure 1 which processes a consonant with formants
- Figure 4 shows the part of the event modifier of Figure 1 which deals with an explosion consonant;
- Figure 5 shows the part of the event modifier of Figure 1 which deals with a friction consonant;
- Figure 6 shows the part of the event modifier in Figure 1 that deals with transitions
- Figure 7 shows the part of the event modifier of Figure 1 which deals with speech silence or noise;
- Figure 8 shows a variant of part of the processing circuit;
FIG. 9 is analogous to FIG. 2 to which the variant according to FIG. 8 has been added.

 FIG. 1 represents the functional block diagram of the processing device according to the present invention. The processing device comprises an event detector: vowel, consonant, transitions, silence of speech or noise, intended to segment the signal on the time axis, so that each type of event is then sent in a circuit of event-specific treatment, each relevant index of which will be specifically modified in terms of time-energy frequencies to be adapted to the perception of the deaf subject.

 The role of the event detector is to recognize the various events which occur over time and which are a vowel, transitions, a consonant consonant, a friction consonant, an explosion consonant, a silence of speech (or noise ) greater than 300 ms.

 The event detector consists of an energy analyzer 11 which records the intensity curve and when it is at a minimum, medium, maximum, suddenly variable level, or at a level of speech silence (or presence of noise), a time structure analyzer 12 which records which sequences are where the event is continuous and those where it is of the explosion type, a frequency structure analyzer 13 which records the sequences where the event has a harmonic frequency structure, composed of formants, and those where the event has a frequency structure of random type (friction noise, explosion noise).

 The signal 10 is sent simultaneously to the three analyzers which send their information to an operator detector 14 which synthesizes it and determines which of the events is in progress. The type of event is selected by the following characteristics: a vowel by a continuous temporal structure, a maximum energy, a harmonic frequency structure (formants) the transitions by their continuous temporal structure, a rising energy at the beginning of a transition, a descending energy at the end of a transition, their harmonic frequency structure (formants) a consonant with formants by a continuous temporal structure, an average energy, a harmonic frequency structure (formants) a friction consonant by a continuous temporal structure, a structure random frequency, medium energy; an explosion consonant by an interrupted temporal structure, a random frequency structure (explosion), a minimum energy.

 The signal is then routed by a line 18 to an input time segmenter 15 whose role is to cut the current event to separate it from the event following the current event is then sent by line 20 to a event selector 16 under control 17 of the operator detector 14.

 The role of the event selector is to orient the current event according to its type towards its own specific processing circuit.

 For its processing, the signal is sent, via line 26, to an event modifier 21. The event modifier 21 includes units for modifying the energy 21 a, the frequency structure 21 b and the structure time 21 c, each unit comprising a set of six circuits, each specializing in the processing of one of the types of events.

 A time offset incrementor 22, connected at 24 to the time modification unit 21c, records the lengthening or shortening time resulting from the time processing of an event; this incrementer reduces speech or noise silences greater than 300 ms in the speech of a speaker to cancel each time there is speech delay caused by the signal processing.

 The output 27 of the event modifier 21 is connected to an adder of output time sequences 23 which puts back end to end, in time, the events after they have each undergone a specific signal processing; the signal is then routed to output 25 of the processing device according to the invention.

 We will now describe, in more detail, the event modifier 21 and its operation.

 Figure 2 deals with the case of a vowel; when a vowel type event is detected, the event selector 16 sends this signal, to the input of the vowel processing device of the modifier 21, in four bandpass filters F1, F2, F3, F4; the filter F1 has the role of isolating the laryngeal fundamental; the setting 212 makes it possible, depending on the deafness, to attenuate the amplitude of the laryngeal fundamental from 0 to - 30 dB because it is a weakly relevant index in the perception of vowels; the filter F2 has the role of frequently isolating the first spectral enhancement RF1 of the vowel; depending on the vowel, the RF1 has a value between 250 and 750 Hz, this is why the filter F2 is configured to let this frequency band pass; the filter F3 has the role of frequently isolating RF2 from the vowel; depending on the vowel, the RF2 has a value between 750 and 2400 Hz, hence the frequency values of the filter F3; the role of the filter F4 is to let pass the spectral components of the vowel greater than 2400 Hz; the frequency values of filters F2, F3, F4, valid for the French language and many foreign languages, may vary slightly depending on the other languages; the setting 219 makes it possible to attenuate the amplitude of these formants from 0 to - 30 dB depending on the deafness, since these are weakly relevant indices in the perception of vowels.

 The RFI and the RF2 of the vowel, passing respectively through filters F2 and F3, will each undergo a processing of their frequency envelope to be better perceived by the deaf; these two formants indeed constitute the two relevant indices characterizing a given vowel. The frequency processing of RF1 must be regulated separately from that of RF2 since the perception problems are generally not the same in their respective frequency zone; this is why there are two processing circuits, one for RF1 and the other for RF2.

 Each of these two circuits is composed of an adjustable dynamic expander 201, 203 (202, 204) to increase the amplitude of the form and give the deaf ear more ease in having a fair feeling of frequency height of the form in question.

 At the output of the dynamic expander, the trainer is sent to a max detector 208, 209 which determines the most intense frequency of the trainer's frequency envelope. This detector is coupled to an adder of frequencies lower 210, 215 at the most intense frequency and to an adder 211, 216 of frequencies higher than the most intense frequency; the role of the adder 210, 215 is to stretch the frequency envelope of the form to the low frequencies, that of the adder 211, 216 is to stretch the frequency envelope of the form to the high frequencies. with an adjustment 213, 214, 217, 218 determining the slope of the envelope from the maximum detected, which is adjustable by an attenuation and makes it possible to favor the stimulation of frequency zones less disturbed than that of the formant in a deaf person given, and therefore to promote its perception.

 The assembly constituted by the max detector 208, 209, the frequency adders lower 210, 215 and higher 211, 216 provided with the adjustments 213, 214, 217, 218 constitutes, as it will have been understood, an envelope transformer, here the envelope of the outgoing signal from the expander 201, 203.

 All of the signals coming out of the filters, of the dynamic expanders via the lines 205, 206 and of the frequency adders are added by a frequency adder 220 to reconstruct an entire and deformed spectrum of the vowel. The signal 222 is then sent to a time reducer 221 adjustable by 224, which will reduce the effective duration of the vowel from 0 to 40%: in fact, obtaining a shorter vowel will attenuate the phenomenon of temporal masking of this vowel on the following events in time.

The reduction values for the duration of the vowel make it possible not to alter the perception thereof, the resulting duration being sufficient for correct perception.

 FIG. 3 deals with the case of a formant consonant when an event of the formant consonant type is detected, the event selector 16 sends this signal to the input of the device for processing consonants to formants of the modifier 21; the processing is identical to that of vowels with the following differences linked to a problem of perception specific to consonants with formants the choice of the amplitude of each relevant index composing the consonant with formants and the choice of the resulting frequency envelope of each forming are generally different from those of vowels.

 The elements of FIG. 3 playing the same role as those of FIG. 2 have been identified by a number in 300 instead of a number in 200.

 An extension operator 322 will not shorten but lengthen the duration of the consonant to form to facilitate perception since this type of consonant is frequently frowned upon by the deaf ear. The duration of the extension is selected by a setting 324; it can be increased from 0 to 40% depending on the deafness.

 Figure 4 deals with the case of an explosion consonant; when an explosion type event is detected, the event selector 16 sends this signal to the input of the explosion treatment processing circuit of the modifier 21; the event is sent in two band-pass filters: the CF1 filter intended to allow the possible laryngeal fundamental of the consonant to pass, the CF5 high-pass filter for frequencies above 400 Hz intended to allow the explosion of the consonant.

 The amplitude of the possible fundamental is chosen by the 412 setting from 0 to + 30 dB to allow a more pronounced perception of it. It is then sent to an extension operator 421 which increases the duration of the consonant using a setting 424 from 0 to 40%.

 After having passed through the filter CF5, the frequency envelope of the explosion is treated in the same way as the forming of a vowel, except that the settings 402, 413, 414 will be different to adapt the spectral envelope of the explosion at the perception of the deaf subject. Stretching the spectral envelope towards the treble will decrease the temporal trail of the explosion while stretching the spectral envelope towards the bass will have the effect of increasing the temporal halftone of the explosion.

 The new shape of the frequency envelope of the explosion, after having been reconstructed and added in 432 to a possible fundamental by a frequency adder 420, is sent in 422 in a retarder 425 controlled in 434 by the extension operator 421 and delaying the production of the explosion.

 FIG. 5 deals with the case of a friction consonant when a friction consonant is detected, it is addressed by the event selector 16 to the friction consonant processing circuit of the modifier 21; as with the explosion consonant, a possible laryngeal fundamental is isolated by a filter CF1, the amplitude of which can be increased by a setting 412 from 0 to 30 dB. Friction noise passes through the CF5 filter; its frequency envelope is treated as the shape of a vowel, except that it is random noise and that the settings 402, 413, 414 are chosen to adapt the frequency envelope of the friction to the deaf subject so that 'he can perceive it better.

 After having been reconstructed by a frequency adder 420, the signal is sent at 436 to an extension operator 421; the increase in the duration of the event is selected from 0 to 40% by setting 424 depending on the deafness.

 Figure 6 deals with the case of transitions; once detected, the transitions type event is sent by the event selector in the transitions processing circuit of the modifier 21 a first temporal modification of the signal is carried out by a phase insertion operator 500; this adds one or more additional phases (setting 531 depending on the deafness) between two phases of the input signal; the added phase or phases have a spectrum corresponding to the average of the spectra of the preceding and following phases. This operation has the effect of lengthening each transition over time and softening the slope of each of them. Each transition will thus be more easily perceived by the deaf ear.

 The signal is then sent to a device which allows processing of the frequency envelope of the transition of the first forming TF2 and that of the transition of the second forming TF3; at all times, this processing of the frequency envelope is identical to that of the Rlil and the RF2 of the vowels, with however a difference in the settings 502, 504, 504, 513, 514, 517, 518, as well as the settings 512 and 519, this in order to specifically adapt the shape and energy of each transition so that it is better perceived by the deaf ear.

 Each transition is sent to a slope and extension calculator 525, 526, 527 which calculates the frequency variation over time of the transition and determines which extension must be added to it to make its character of frequency variation more marked.

 An extension operator 521 adds to each transition the time segment determined by the slope and extension calculator. The length of the extension to be added to the transitions is chosen using setting 524.

 FIG. 7 deals with the case of the absence of speech, therefore of the presence of noises during this period; when speech silence greater than 300 ms is detected, the event selector sends this signal to an adjustable attenuator 223 to reduce the intensity of the noise, then to a time reducer 221 controlled by the time offset incrementor 22. Indeed, the latter recorded the delay accumulated over the events by the signal processing and determines the delay to be made up compared to the strictly real time of pronunciation of words by a speaker.

 When the temporal reducer subtracts from the duration of the speech silence the delay accumulated by the successive treatments, the temporal offset incrementer is reset to zero if, of course, the duration of the delay is less than that of the absence of speech.

 Referring to FIG. 8, in order to better exploit the least disturbed frequency zones, for example for a given deaf ear, it is advantageous to add, in the processing circuits, to the output 103 of the dynamic expander 102 , a frequency envelope detector 104; the frequencies of the envelope are then divided into 107 by an integer n, for example 2, and multiplied in 108 by an integer n ', for example also 2, then sent to the frequency adder 114; adjustments 109 and 110 allow adaptation to deafness. The whole of this particular arrangement is noted 100 in FIG. 8.

 By applying, for example, this disposition to the processing of vowels, the corresponding synoptic is shown in FIG. 9, which is analogous to FIG. 2, but to which two dispositions have been added 100; it will be noted that, of course, the arrangement 100 applied, in this example, to the processing of vowels, can also, advantageously, be applied to other processing operations as described above.

 The envelope transformer, as defined with reference to FIG. 2, and shown diagrammatically, in this FIG. 9, by the dashed line designated by the reference 600, can, as a variant, be used to transform any envelope of the signal at some level whatever processing, in particular after division into 107 and / or multiplication into 108 according to the arrangement noted 100 in FIG. 8; such a variant is shown diagrammatically in this FIG. 8 by the dashed lines denoted 600; this arrangement constitutes an additional means of adapting the envelope of the output signal to the hearing abilities of the subject being treated.

 The treatment device according to the invention has been described in application to the correction of deafness; but, of course, it is not limited to this application; it can also be used in speech perception in noise, speech therapy, foreign language learning, radio systems, telecommunications, cochlear implants, speech signal processing of divers, amplifiers (music, singing, film sounds ... ).

Claims (15)

 1. Device for processing a sound signal (10) comprising an event detector, capable of recognizing the various sound phenomena and comprising an analyzer (12) of the temporal structure of each event to distinguish the sequences where said event is continuous. those where it is of the explosion type, and an event modifier (21) comprising a modification unit (21c) for modifying said temporal structure.  2. Processing device according to claim 1 characterized in that the information delivered by said analyzer (12) is sent to an operator detector (14) which determines the type of event in progress.  3. Treatment device according to claim 2, characterized in that the type of event in progress is chosen from the group formed by a vowel, a consonant consonant, a friction consonant, an explosion consonant, a transition , silence of speech, during which noise will be present, and all sound carried by a signal.  4. Processing device according to one of claims 2 or 3, characterized in that two consecutive events are separated from each other by a time segmenter (15).  5. Processing device according to one of claims 2 to 4, characterized in that the event modifier (21) comprises a specific processing unit provided for each of the event types.  6. Processing device according to claim 5, characterized in that an event selector (16) directs the event to its specific processing unit.  7. Processing device according to claims 2 and 6, characterized in that the orientation of the event by the event selector is controlled (17) by the operator detector (14).  8. Processing device according to claims 5 and 7, characterized in that the specific processing unit comprises six specialized processing circuits for processing vowels, formants, friction and explosion consonants, transitions and speech silences.  9. Processing device according to one of claims 1 to 8, characterized in that the event detector also comprises an energy analyzer (11) of the signal of each event, said event modifier (21) further comprising a modification unit (21a) for modifying the energy of at least one of the events.  10. Processing device according to one of claims 1 to 8, characterized in that the event detector also comprises a frequency structure analyzer (13) for distinguishing the sequences where said event has a harmonic structure, composed of formants , those where it has a random structure, said event modifier (21) further comprising a modification unit (21b) for modifying the frequency structure of at least one of the events.  11. Processing device according to claims 1, 9 and 10, characterized in that the event modifier (21) comprises three modification units (21a, 21b, 21c), to modify the energy, the time structure, and the frequency structure of at least one of the events.  12. Processing device according to claim 1, characterized in that the modification unit (21c) of the temporal structure is connected to a temporal offset incrementor (22) which records the lengthening or shortening time resulting from modification of the time structure.  13. Processing device according to claim 12, characterized in that the incrementer (22) reduces the speech silence greater than a given value.  14. Processing device according to one of claims 1 to 13, characterized in that the events processed are reclassified chronologically in an adder (23) of time sequences whose output (25) constitutes the output signal of the processing device .  15. Apparatus comprising means for receiving a sound signal, means for processing said signal and means for restoring the modified sound signal, characterized in that the means for processing said signal comprise the processing device according to claims 1 to previous 14.