CN109493877A - A kind of sound enhancement method and device of auditory prosthesis - Google Patents
A kind of sound enhancement method and device of auditory prosthesis Download PDFInfo
- Publication number
- CN109493877A CN109493877A CN201710817728.7A CN201710817728A CN109493877A CN 109493877 A CN109493877 A CN 109493877A CN 201710817728 A CN201710817728 A CN 201710817728A CN 109493877 A CN109493877 A CN 109493877A
- Authority
- CN
- China
- Prior art keywords
- audio data
- sub
- band
- sound
- acoustic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 39
- 239000000284 extract Substances 0.000 claims abstract description 3
- 230000001629 suppression Effects 0.000 claims description 25
- 238000001914 filtration Methods 0.000 claims description 24
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 23
- 238000006243 chemical reaction Methods 0.000 claims description 18
- 230000006835 compression Effects 0.000 claims description 18
- 238000007906 compression Methods 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 18
- 230000009467 reduction Effects 0.000 claims description 18
- 230000003321 amplification Effects 0.000 claims description 17
- 238000010183 spectrum analysis Methods 0.000 claims description 13
- 230000008030 elimination Effects 0.000 claims description 10
- 238000003379 elimination reaction Methods 0.000 claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 9
- 230000007613 environmental effect Effects 0.000 claims description 7
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 238000003786 synthesis reaction Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000004807 localization Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 3
- 230000005236 sound signal Effects 0.000 abstract description 2
- 239000004568 cement Substances 0.000 abstract 2
- 230000006870 function Effects 0.000 description 6
- 208000016354 hearing loss disease Diseases 0.000 description 6
- 206010011878 Deafness Diseases 0.000 description 5
- 230000010370 hearing loss Effects 0.000 description 5
- 231100000888 hearing loss Toxicity 0.000 description 5
- 230000003595 spectral effect Effects 0.000 description 5
- 238000001228 spectrum Methods 0.000 description 3
- 208000032041 Hearing impaired Diseases 0.000 description 2
- 230000002238 attenuated effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000012076 audiometry Methods 0.000 description 1
- 210000000721 basilar membrane Anatomy 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0224—Processing in the time domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Otolaryngology (AREA)
- Neurosurgery (AREA)
- General Health & Medical Sciences (AREA)
- Prostheses (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The embodiment of the invention discloses a kind of sound enhancement method of auditory prosthesis and devices, are related to medical electronic technology and Audio Signal Processing field.Method in the embodiment of the present invention includes: the four-way audio data for obtaining auditory prosthesis；The audio data that will acquire extracts acoustic enviroment feature, obtains the corresponding acoustics scene of the audio data；The compensation of subchannel sound and speech enhan-cement are carried out to the audio data got according to the acoustics scene；Exporting two-way enhances audio data.The audio data that terminal will acquire carries out speech enhan-cement processing, final output two-way realaudio data on portable terminal.Intelligence improves sound quality, substantially increase the prevalence of hearing aid fits, it can achieve better hearing aid effect and enhancement method, simultaneously because audio data processing is solidificated on auditory prosthesis processor, but the general processor chip based on portable terminal, conducive to the perfect of following system upgrade and sound enhancement method.
Description
Technical Field
The present invention relates to the field of medical electronic technology and audio signal processing, and in particular, to a method and apparatus for speech enhancement in a hearing aid device.
Background
Nowadays, China enters an age-accelerating social age, the life expectancy of old people is improved, the number of people with hearing loss and damage caused by excessive application of electronic products is on the rise, and in recent years, with the improvement of medical care level, the proportion of old people wearing hearing aids and hearing-impaired patients is increasing. Today, hearing aid technology is based on advanced digital signal processing, wireless communication and artificial intelligence techniques. With the rapid development of technology, the size of hearing aids is getting smaller and smaller, and the functions of hearing aids are getting more and more comprehensive, such as multi-channel wide dynamic range compression, active noise reduction, adaptive directivity, sound field analysis and wireless connection to other audio or communication systems.
An important aspect of hearing aids is to ensure that the hearing loss of a hearing patient is compensated and the audio quality is improved without further loss of hearing. The existing hearing aid built-in algorithm is solidified in a processor and cannot be intelligently upgraded along with the change of the processor.
Disclosure of Invention
In order to solve the above technical problem, embodiments of the present invention provide a method and an apparatus for speech enhancement of a hearing aid device, which utilize a portable intelligent terminal (e.g., a mobile phone) to implement a corresponding speech enhancement function of a hearing aid function.
In a first aspect, the present invention provides a method for speech enhancement of a hearing device, comprising:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
Preferably, extracting the acoustic environment features from the acquired audio data, and obtaining the acoustic scene corresponding to the audio data includes:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
Preferably, the performing of the channel-division sound compensation and the speech enhancement on the acquired audio data according to the acoustic scene includes:
preprocessing the audio data and filtering the audio data in channels;
sub-band division is carried out on the audio data after the channel division filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
gating a sound source corresponding to the audio data according to the determined environment mode, and calculating the angle of the position of the sound source;
according to the determined angle of the position of the sound source and the signal to noise ratio of the sub-band, carrying out noise reduction and howling elimination processing on each sub-band of the audio data;
performing dynamic compression and sound intensity amplification processing on each sub-band of the audio data subjected to noise reduction;
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performing linear phase compensation;
each sub-band of the audio data is combined into a time-domain speech signal.
Preferably, the pre-processing the audio data comprises:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
Preferably, extracting the acoustic environment features from the acquired audio data, and obtaining the acoustic scene corresponding to the audio data further includes:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
Preferably, the sound source corresponding to the audio data is gated according to the determined environment mode, and calculating the angle of the position of the sound source includes:
gating sound sources of all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
Preferably, the noise reduction processing for each sub-band of the audio data comprises:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
Preferably, the performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performing linear phase compensation includes:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
In a second aspect, the present invention also provides a speech enhancement device for a hearing device, comprising:
a sound pickup module configured to acquire four-channel audio data of the hearing aid device;
the acoustic environment monitoring module is configured to extract acoustic environment characteristics from the acquired audio data and acquire an acoustic scene corresponding to the audio data;
the sound processing module is arranged for performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and the output module is used for outputting two paths of enhanced audio data.
Preferably, the extracting, by the acoustic environment monitoring module, the acoustic environment feature from the acquired audio data, and the obtaining an acoustic scene corresponding to the audio data includes:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
Preferably, the sound processing module includes:
a preprocessing unit configured to perform preprocessing and channel-wise filtering on the audio data;
the sub-band dividing unit is used for dividing the sub-band of the audio data after the sub-channel filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
the sound source positioning unit is used for gating a sound source corresponding to the audio data according to the determined environment mode and calculating the angle of the position of the sound source;
the howling suppression and feedback elimination unit is set to perform noise reduction and howling elimination processing on each sub-band of the audio data according to the determined angle of the position of the sound source and the signal-to-noise ratio of the sub-band;
the compression and amplification unit is used for carrying out dynamic compression and sound intensity amplification processing on each sub-band of the audio data subjected to noise reduction;
the sound compensation unit is used for performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data and performing linear phase compensation;
a sound synthesis unit arranged to combine each sub-band of the audio data into a time domain speech signal.
Preferably, the preprocessing module preprocessing the audio data includes:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
Preferably, the acoustic environment monitoring module is further configured to:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
Preferably, the sound source positioning unit gates the sound source corresponding to the audio data according to the determined environment mode, and calculating the angle of the position of the sound source includes:
selecting sound sources in all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
Preferably, the performing, by the howling suppression and feedback cancellation unit, noise reduction processing on each sub-band of the audio data includes:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
Preferably, the sound compensation unit performs time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performs linear phase compensation, including:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
In a third aspect, the present invention further provides a speech enhancement apparatus, including: a memory and a processor;
the memory is used for storing executable instructions;
the processor is configured to execute the executable instructions stored in the memory, and perform the following operations:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
In a fourth aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the following operations are performed:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
According to the voice enhancement method and device of the hearing aid device, the hearing aid device on the ear side is used for acquiring four paths of audio data and transmitting the four paths of audio data to the portable terminal, the acquired audio data are subjected to voice enhancement processing on the portable terminal, and finally two paths of real-time audio data are output. Different from a hearing aid system in a common hearing aid device, the embodiment of the invention fully considers different performance conditions of a portable terminal processor and provides a voice hearing aid enhancement method for intelligent upgrading under different portable terminals. The tone quality is intelligently improved, the universal degree of wearing of the hearing aid is greatly improved, a better hearing aid effect and an enhancement mode can be achieved, and meanwhile, because the audio data processing is not solidified on a processor of the hearing aid device but is based on a general processor chip of a portable terminal, the improvement of a future system upgrading and a voice enhancement method is facilitated.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
Fig. 1 is a flowchart of a speech enhancement method of a hearing aid device according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a speech enhancement device of a hearing aid device according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a sound processing module according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
Nowadays, portable intelligent terminals (such as mobile phones) have higher popularity and the computing power of general processors of portable terminals has also been stronger. However, most of the existing hearing aids do not have a perfect corresponding voice enhancement implementation method for matching portable terminals, which can implement hearing aid functions, and the existing hearing aid built-in algorithm is solidified in a processor and cannot be intelligently upgraded along with the change of the processor. As shown in fig. 1, an embodiment of the present invention provides a method for enhancing speech of a hearing aid device, implemented by a portable terminal processor, including:
s101, acquiring four-channel audio data of a hearing aid device;
s102, extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
s103, performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and S104, outputting two paths of enhanced audio data.
The four-channel audio data in the embodiment of the present invention refers to: the portable terminal acquires sound input of a left-ear front microphone and sound input of a left-ear rear microphone, and sound input of a right-ear front microphone and sound input of a right-ear rear microphone of the hearing aid device.
According to the embodiment of the invention, on the premise of realizing intelligent voice enhancement based on the portable terminal and not updating the hearing aid device at the ear side, the hearing loss of the hearing-impaired patient can be compensated by only using the portable terminal, four paths of audio data are acquired by the hearing aid device at the ear side and transmitted to the portable terminal, the acquired audio data are subjected to voice enhancement processing on the portable terminal, and finally two paths of real-time audio data are output. Tone quality is improved to intelligence, has improved the general degree that the audiphone was worn greatly. The intelligent voice enhancement can be carried out in real time while the basic voice hearing aid function is completed, so that the intelligent voice enhancement is convenient for a user to use and is convenient for product upgrading.
Step S102, extracting acoustic environment characteristics from the acquired audio data, and acquiring an acoustic scene corresponding to the audio data includes:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
Step S103 of performing the channel-by-channel sound compensation and the speech enhancement on the acquired audio data according to the acoustic scene includes:
s1031, preprocessing the audio data and filtering the audio data in a channel division manner;
s1032, sub-band division is carried out on the audio data after the channel division filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
s1033, gating a sound source corresponding to the audio data according to the determined environment mode, and calculating the angle of the position of the sound source;
s1034, according to the determined angle of the position of the sound source and the signal-to-noise ratio of the sub-band, carrying out noise reduction and howling elimination processing on each sub-band of the audio data;
s1035, performing dynamic compression and sound intensity amplification processing on each sub-band of the audio data subjected to noise reduction;
s1036, performing time-frequency conversion on the frequency domain signals corresponding to each sub-band of the compressed and amplified audio data, and performing linear phase compensation;
and S1037, combining each sub-band of the audio data into a time domain voice signal.
In step S1031 in this embodiment, the branch channel filtering is performed, the background noise is determined according to the detected voice endpoint, and the first filtering is performed by using the spectral subtraction method, so as to obtain four paths of voice signals from which the noise is primarily removed.
Wherein preprocessing the audio data comprises:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
In the embodiment of the invention, the preprocessing mainly refers to the processing of pre-emphasis, and the high-pass filtering of a first-order filter is carried out on the high-frequency component, so that the high-frequency resolution of the voice is increased.
The embodiment of the invention carries out sub-channel filtering through the gamma filters, and the process is as follows:
the cochlear basilar membrane has different frequency specificities at different locations, depending on the specifics of the human ear architecture. This specificity can be expressed in terms of an n-th order non-uniform width gamma-atom filter whose time domain expression satisfies the following formula, i.e.,
wherein,representing the phase, fc the center frequency, b the bandwidth, N the order of the filter, t the time, a the amplitude.
The noise signals contained in the audio data from four channels usually exist in the low frequency band, and the noise is attenuated by using spectral subtraction and variable noise subtraction parameters α, so that the degree of speech distortion is controllable, for the speech signals in the high frequency band, the noise spectral components in the high frequency band are removed by using a cross-correlation function method, the relevant parameters required for positioning are retained, and the speech signals are not attenuated, wherein the determination of the variable noise subtraction parameters α can be obtained according to the following formula:
where k represents the subband sequence number, l represents the subband frame number,is a random initial value, SNR, representing αpSignal to noise ratio value representing a posterioriσ is a positive integer for controlling the extent of spectral subtraction of the sub-band noise spectrum, β and αi(k) The most significant is related to the estimate of the a priori SNR and is the evaluation parameter, β is used to prevent the denominator from being zero (there may be cases where the a posteriori SNR goes to zero), and β is calculated as the reciprocal of the maximum-minimum difference of α from the speech segments.
The division between the high band and the low band is calculated by the noise power spectrum of each subband output signal, and the division frequency range is usually selected to be about 800Hz to 1000 Hz.
Step S102 is to extract acoustic environment features from the acquired audio data, and after obtaining an acoustic scene corresponding to the audio data, the method further includes:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
Wherein the directivity control parameters include: and parameters such as a binaural time difference, a binaural intensity difference, a binaural phase difference, a front-back ear phase difference and the like.
Gating the sound source corresponding to the audio data according to the determined environment mode, wherein the step of calculating the angle of the position of the sound source comprises the following steps:
gating sound sources of all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
Performing noise reduction processing on each sub-band of the audio data includes:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
In the embodiment of the invention, the modulation amplitude parameter is determined by the environment, because the envelope of the voice signal has modulation characteristics, after the spectrum analysis, the modulation rate can be used for identifying whether the input acoustic signal is voice or noise according to the size of the modulation rate, and the noise suppression parameter is different according to the environment, the noise spectrum of the environment in noisy and quiet environments, and the calculated input signal-to-noise ratio is also different, so that the noise suppression parameter is used for calculating the variable noise subtraction parameter α.
Performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performing linear phase compensation comprises:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
The compression and amplification scale parameter is determined by the hearing loss condition of the patient, a hearing loss graph is generated after audiometry, the hearing conditions of the patient under different frequencies are marked, the compression and amplification scale parameter is determined according to the data to amplify the data to the normal hearing level degree, and in sound compensation, the compression and amplification scale coefficients under different environments are different to compensate in different degrees.
According to the embodiment of the invention, voice signal integration is performed after the subsequent processing of pre-emphasis filtering, gamma-atom sub-channel filtering, spectral subtraction and the like. Can achieve better hearing aid effect and enhancement.
As shown in fig. 2, an embodiment of the present invention further provides a speech enhancement device for a hearing aid device, disposed at a portable terminal side, including:
a sound pickup module 11 configured to acquire four-channel audio data of the hearing aid device;
the acoustic environment monitoring module 12 is configured to extract acoustic environment features from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
the sound processing module 13 is configured to perform channel-division sound compensation and speech enhancement on the acquired audio data according to the acoustic scene;
an output module 14 configured to output two paths of enhanced audio data.
The acoustic environment monitoring module 12 extracts acoustic environment features from the acquired audio data, and acquiring an acoustic scene corresponding to the audio data includes:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
The sound processing module 13 includes:
a preprocessing unit 131 configured to perform preprocessing and channel-wise filtering on the audio data;
a sub-band division unit 132 configured to sub-band-divide the audio data after the channel division filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
a sound source positioning unit 133 configured to gate a sound source corresponding to the audio data according to the determined environment mode, and calculate an angle of a position where the sound source is located;
a howling suppression and feedback elimination unit 134 configured to perform noise reduction and howling elimination processing on each sub-band of the audio data according to the determined angle of the position of the sound source and the signal-to-noise ratio of the sub-band;
a compressing and amplifying unit 135 configured to perform dynamic compression and sound intensity amplification processing on each sub-band of the audio data after noise reduction;
the sound compensation unit 136 is configured to perform time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and perform linear phase compensation;
a sound synthesis unit 137 arranged to combine each sub-band of the audio data into a time domain speech signal.
The preprocessing module preprocessing the audio data comprises:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
The acoustic environment monitoring module is further configured to:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
The sound source positioning unit 133 gates the sound source corresponding to the audio data according to the determined environment mode, and calculating the angle of the position of the sound source includes:
selecting sound sources in all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
The noise reduction processing of each sub-band of the audio data by the howling suppression and feedback elimination unit comprises:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
The sound compensation unit performs time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performs linear phase compensation, including:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
An embodiment of the present invention further provides a speech enhancement apparatus, including: a memory and a processor;
the memory is used for storing executable instructions;
the processor is configured to execute the executable instructions stored in the memory, and perform the following operations:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
The embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the following operations are performed:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
It will be understood by those skilled in the art that all or part of the steps of the above methods may be implemented by a program instructing associated hardware (e.g., a processor) which may be stored in a computer readable storage medium such as a read only memory, a magnetic or optical disk, etc. Alternatively, all or part of the steps of the above embodiments may be implemented using one or more integrated circuits. Accordingly, the modules/units in the above embodiments may be implemented in hardware, for example, by an integrated circuit, or may be implemented in software, for example, by a processor executing programs/instructions stored in a memory to implement the corresponding functions. Embodiments of the invention are not limited to any specific form of hardware or software combination.
Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (18)
1. A method for speech enhancement of a hearing assistance device, comprising:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
2. The speech enhancement method according to claim 1, wherein extracting acoustic environment features from the acquired audio data and obtaining an acoustic scene corresponding to the audio data comprises:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
3. The speech enhancement method of claim 2, wherein performing the sub-channel sound compensation and speech enhancement on the acquired audio data according to the acoustic scene comprises:
preprocessing the audio data and filtering the audio data in channels;
sub-band division is carried out on the audio data after the channel division filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
gating a sound source corresponding to the audio data according to the determined environment mode, and calculating the angle of the position of the sound source;
according to the determined angle of the position of the sound source and the signal to noise ratio of the sub-band, carrying out noise reduction and howling elimination processing on each sub-band of the audio data;
performing dynamic compression and sound intensity amplification processing on each sub-band of the audio data subjected to noise reduction;
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performing linear phase compensation;
each sub-band of the audio data is combined into a time-domain speech signal.
4. The speech enhancement method of claim 3 wherein pre-processing the audio data comprises:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
5. The speech enhancement method according to claim 3, wherein the step of extracting the acoustic environment features from the obtained audio data and obtaining the acoustic scene corresponding to the audio data further comprises:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
6. The speech enhancement method of claim 5, wherein the gating of the sound source corresponding to the audio data according to the determined environmental mode, the calculating of the angle of the position of the sound source comprises:
gating sound sources of all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
7. The speech enhancement method of claim 5 wherein denoising each sub-band of the audio data comprises:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
8. The speech enhancement method of claim 5, wherein performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data and performing linear phase compensation comprises:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
9. A speech enhancement device for a hearing assistance device, comprising:
a sound pickup module configured to acquire four-channel audio data of the hearing aid device;
the acoustic environment monitoring module is configured to extract acoustic environment characteristics from the acquired audio data and acquire an acoustic scene corresponding to the audio data;
the sound processing module is arranged for performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and the output module is used for outputting two paths of enhanced audio data.
10. The speech enhancement device according to claim 9, wherein the acoustic environment monitoring module extracts acoustic environment features from the acquired audio data, and obtaining an acoustic scene corresponding to the audio data includes:
extracting acoustic environment features of the audio data;
and matching the extracted acoustic environment characteristics with a preset voice environment, and determining the environment mode of the user.
11. The speech enhancement device of claim 10 wherein the sound processing module comprises:
a preprocessing unit configured to perform preprocessing and channel-wise filtering on the audio data;
the sub-band dividing unit is used for dividing the sub-band of the audio data after the sub-channel filtering; carrying out spectrum analysis on the sub-band of each audio data to obtain the signal-to-noise ratio of the sub-band of the audio data;
the sound source positioning unit is used for gating a sound source corresponding to the audio data according to the determined environment mode and calculating the angle of the position of the sound source;
the howling suppression and feedback elimination unit is set to perform noise reduction and howling elimination processing on each sub-band of the audio data according to the determined angle of the position of the sound source and the signal-to-noise ratio of the sub-band;
the compression and amplification unit is used for carrying out dynamic compression and sound intensity amplification processing on each sub-band of the audio data subjected to noise reduction;
the sound compensation unit is used for performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data and performing linear phase compensation;
a sound synthesis unit arranged to combine each sub-band of the audio data into a time domain speech signal.
12. The speech enhancement device of claim 11, wherein the pre-processing module pre-processes the audio data comprising:
and performing first-order high-pass filtering on the components of the audio data, the frequencies of which are greater than a preset value.
13. The speech enhancement device of claim 11, wherein the acoustic environment monitoring module is further configured to:
obtaining parameters of at least one of the following of the environmental patterns:
a modulation amplitude parameter, a directivity control parameter, a compression-amplification ratio parameter, and a noise suppression parameter.
14. The apparatus according to claim 13, wherein the sound source localization unit gates the sound source corresponding to the audio data according to the determined environment mode, and the calculating the angle of the position of the sound source comprises:
selecting sound sources in all directions of the hearing aid device according to the directivity control parameters;
the angle of the position of the sound source is calculated.
15. The apparatus of claim 13, wherein the means for performing noise reduction on each sub-band of the audio data comprises:
identifying whether the audio data is noise according to the modulation amplitude parameter based on the envelope modulation characteristic and the spectrum analysis result of the audio data;
and carrying out suppression processing on the noise according to the determined signal-to-noise ratio and the noise suppression parameter.
16. The apparatus of claim 13, wherein the sound compensation unit performs time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data, and performs linear phase compensation, and comprises:
performing time-frequency conversion on the frequency domain signal corresponding to each sub-band of the compressed and amplified audio data;
and performing phase compensation of corresponding degree according to the compression and amplification scale factor.
17. A speech enhancement apparatus, comprising: a memory and a processor;
the memory is used for storing executable instructions;
the processor is configured to execute the executable instructions stored in the memory, and perform the following operations:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
18. A computer-readable storage medium having computer-executable instructions stored thereon that, when executed, perform operations comprising:
acquiring four-channel audio data of a hearing aid device;
extracting acoustic environment characteristics from the acquired audio data to obtain an acoustic scene corresponding to the audio data;
performing channel-division sound compensation and voice enhancement on the acquired audio data according to the acoustic scene;
and outputting two paths of enhanced audio data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710817728.7A CN109493877B (en) | 2017-09-12 | 2017-09-12 | Voice enhancement method and device of hearing aid device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710817728.7A CN109493877B (en) | 2017-09-12 | 2017-09-12 | Voice enhancement method and device of hearing aid device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109493877A true CN109493877A (en) | 2019-03-19 |
| CN109493877B CN109493877B (en) | 2022-01-28 |
Family
ID=65688095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710817728.7A Active CN109493877B (en) | 2017-09-12 | 2017-09-12 | Voice enhancement method and device of hearing aid device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109493877B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110022514A (en) * | 2019-05-17 | 2019-07-16 | 深圳市湾区通信技术有限公司 | Noise-reduction method, device, system and the computer storage medium of audio signal |
| CN110728970A (en) * | 2019-09-29 | 2020-01-24 | 华声设计研究院（深圳）有限公司 | Method and device for digital auxiliary sound insulation treatment |
| CN112492495A (en) * | 2019-09-11 | 2021-03-12 | 西万拓私人有限公司 | Method for operating a hearing device and hearing device |
| CN112562265A (en) * | 2020-12-23 | 2021-03-26 | 江苏集萃智能集成电路设计技术研究所有限公司 | Intelligent monitoring system and monitoring method based on hearing aid |
| CN112954569A (en) * | 2021-02-20 | 2021-06-11 | 深圳市智听科技有限公司 | Multi-core hearing aid chip, hearing aid method and hearing aid |
| WO2021189946A1 (en) * | 2020-03-24 | 2021-09-30 | 青岛罗博智慧教育技术有限公司 | Speech enhancement system and method, and handwriting board |
| CN113825082A (en) * | 2021-09-19 | 2021-12-21 | 武汉左点科技有限公司 | Method and device for relieving hearing aid delay |
| WO2022017424A1 (en) * | 2020-07-24 | 2022-01-27 | 华为技术有限公司 | Active noise control method and apparatus, and audio playback device |
| CN115314824A (en) * | 2022-10-12 | 2022-11-08 | 深圳市婕妤达电子有限公司 | Signal processing method and device for hearing aid, electronic equipment and storage medium |
| CN115396800A (en) * | 2022-05-16 | 2022-11-25 | 西安合谱声学科技有限公司 | Directional hearing aid method, directional hearing aid system, hearing aid and storage medium |
| CN113949955B (en) * | 2020-07-16 | 2024-04-09 | Oppo广东移动通信有限公司 | Noise reduction processing method and device, electronic equipment, earphone and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1333994A (en) * | 1998-11-16 | 2002-01-30 | 伊利诺伊大学评议会 | Binaural signal processing techniques |
| US20060206320A1 (en) * | 2005-03-14 | 2006-09-14 | Li Qi P | Apparatus and method for noise reduction and speech enhancement with microphones and loudspeakers |
| CN1967659A (en) * | 2005-11-14 | 2007-05-23 | 北京大学科技开发部 | Speech enhancement method applied to deaf-aid |
| CN101447190A (en) * | 2008-06-25 | 2009-06-03 | 北京大学深圳研究生院 | Voice enhancement method employing combination of nesting-subarray-based post filtering and spectrum-subtraction |
| US20130195302A1 (en) * | 2010-12-08 | 2013-08-01 | Widex A/S | Hearing aid and a method of enhancing speech reproduction |
| CN103686575A (en) * | 2013-11-28 | 2014-03-26 | 清华大学 | Hearing aid |
| CN104038880A (en) * | 2014-06-26 | 2014-09-10 | 南京工程学院 | Method for enhancing voice of double-ear hearing-aid device |
| CN105741849A (en) * | 2016-03-06 | 2016-07-06 | 北京工业大学 | Voice enhancement method for fusing phase estimation and human ear hearing characteristics in digital hearing aid |
-
2017
- 2017-09-12 CN CN201710817728.7A patent/CN109493877B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1333994A (en) * | 1998-11-16 | 2002-01-30 | 伊利诺伊大学评议会 | Binaural signal processing techniques |
| US20060206320A1 (en) * | 2005-03-14 | 2006-09-14 | Li Qi P | Apparatus and method for noise reduction and speech enhancement with microphones and loudspeakers |
| CN1967659A (en) * | 2005-11-14 | 2007-05-23 | 北京大学科技开发部 | Speech enhancement method applied to deaf-aid |
| CN101447190A (en) * | 2008-06-25 | 2009-06-03 | 北京大学深圳研究生院 | Voice enhancement method employing combination of nesting-subarray-based post filtering and spectrum-subtraction |
| US20130195302A1 (en) * | 2010-12-08 | 2013-08-01 | Widex A/S | Hearing aid and a method of enhancing speech reproduction |
| CN103686575A (en) * | 2013-11-28 | 2014-03-26 | 清华大学 | Hearing aid |
| CN104038880A (en) * | 2014-06-26 | 2014-09-10 | 南京工程学院 | Method for enhancing voice of double-ear hearing-aid device |
| CN105741849A (en) * | 2016-03-06 | 2016-07-06 | 北京工业大学 | Voice enhancement method for fusing phase estimation and human ear hearing characteristics in digital hearing aid |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110022514A (en) * | 2019-05-17 | 2019-07-16 | 深圳市湾区通信技术有限公司 | Noise-reduction method, device, system and the computer storage medium of audio signal |
| CN112492495A (en) * | 2019-09-11 | 2021-03-12 | 西万拓私人有限公司 | Method for operating a hearing device and hearing device |
| US11388514B2 (en) | 2019-09-11 | 2022-07-12 | Sivantos Pte. Ltd. | Method for operating a hearing device, and hearing device |
| CN110728970B (en) * | 2019-09-29 | 2022-02-25 | 东莞市中光通信科技有限公司 | Method and device for digital auxiliary sound insulation treatment |
| CN110728970A (en) * | 2019-09-29 | 2020-01-24 | 华声设计研究院（深圳）有限公司 | Method and device for digital auxiliary sound insulation treatment |
| WO2021189946A1 (en) * | 2020-03-24 | 2021-09-30 | 青岛罗博智慧教育技术有限公司 | Speech enhancement system and method, and handwriting board |
| CN113949955B (en) * | 2020-07-16 | 2024-04-09 | Oppo广东移动通信有限公司 | Noise reduction processing method and device, electronic equipment, earphone and storage medium |
| WO2022017424A1 (en) * | 2020-07-24 | 2022-01-27 | 华为技术有限公司 | Active noise control method and apparatus, and audio playback device |
| CN112562265A (en) * | 2020-12-23 | 2021-03-26 | 江苏集萃智能集成电路设计技术研究所有限公司 | Intelligent monitoring system and monitoring method based on hearing aid |
| CN112954569B (en) * | 2021-02-20 | 2022-10-25 | 深圳市智听科技有限公司 | Multi-core hearing aid chip, hearing aid method and hearing aid |
| CN112954569A (en) * | 2021-02-20 | 2021-06-11 | 深圳市智听科技有限公司 | Multi-core hearing aid chip, hearing aid method and hearing aid |
| CN113825082A (en) * | 2021-09-19 | 2021-12-21 | 武汉左点科技有限公司 | Method and device for relieving hearing aid delay |
| CN113825082B (en) * | 2021-09-19 | 2024-06-11 | 武汉左点科技有限公司 | Method and device for relieving hearing aid delay |
| CN115396800A (en) * | 2022-05-16 | 2022-11-25 | 西安合谱声学科技有限公司 | Directional hearing aid method, directional hearing aid system, hearing aid and storage medium |
| CN115314824A (en) * | 2022-10-12 | 2022-11-08 | 深圳市婕妤达电子有限公司 | Signal processing method and device for hearing aid, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109493877B (en) | 2022-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109493877B (en) | Voice enhancement method and device of hearing aid device | |
| CN113676803B (en) | Active noise reduction method and device | |
| US20230352038A1 (en) | Voice activation detecting method of earphones, earphones and storage medium | |
| CN109121057B (en) | Intelligent hearing aid method and system | |
| JP6017825B2 (en) | A microphone and earphone combination audio headset with means for denoising proximity audio signals, especially for "hands-free" telephone systems | |
| CN103236263B (en) | Method, system and mobile terminal for improving call quality | |
| US20060206320A1 (en) | Apparatus and method for noise reduction and speech enhancement with microphones and loudspeakers | |
| US10154353B2 (en) | Monaural speech intelligibility predictor unit, a hearing aid and a binaural hearing system | |
| KR102004460B1 (en) | Digital hearing device using bluetooth circuit and digital signal processing | |
| US9699574B2 (en) | Method of superimposing spatial auditory cues on externally picked-up microphone signals | |
| CN112367600A (en) | Voice processing method and hearing aid system based on mobile terminal | |
| US12096184B2 (en) | Hearing aid comprising a feedback control system | |
| JP2023525138A (en) | Active noise canceling method and apparatus | |
| CN110992967A (en) | Voice signal processing method and device, hearing aid and storage medium | |
| WO2022256577A1 (en) | A method of speech enhancement and a mobile computing device implementing the method | |
| CN113949955A (en) | Noise reduction processing method and device, electronic equipment, earphone and storage medium | |
| CN115314823A (en) | Hearing aid method, system and equipment based on digital sounding chip | |
| Li et al. | A smart binaural hearing aid architecture leveraging a smartphone APP with deep-learning speech enhancement | |
| CN112367599B (en) | Hearing aid system with cloud background support | |
| CN103152686A (en) | Digital hearing-aid with customizable functioning mode and implementation method thereof | |
| US11445307B2 (en) | Personal communication device as a hearing aid with real-time interactive user interface | |
| CN213462323U (en) | Hearing aid system based on mobile terminal | |
| CN109729471A (en) | The ANC denoising device of formula interactive voice earphone is worn for neck | |
| CN114093380B (en) | Voice enhancement method, electronic equipment, chip system and readable storage medium | |
| CN109729463A (en) | The compound audio signal reception device of sound wheat bone wheat of formula interactive voice earphone is worn for neck |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |