CN113038322A - Method and device for enhancing environmental perception by hearing - Google Patents
Method and device for enhancing environmental perception by hearing Download PDFInfo
- Publication number
- CN113038322A CN113038322A CN202110239331.0A CN202110239331A CN113038322A CN 113038322 A CN113038322 A CN 113038322A CN 202110239331 A CN202110239331 A CN 202110239331A CN 113038322 A CN113038322 A CN 113038322A
- Authority
- CN
- China
- Prior art keywords
- audio
- ear
- sound
- transfer function
- generated
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000008447 perception Effects 0.000 title claims abstract description 31
- 230000002708 enhancing effect Effects 0.000 title claims abstract description 29
- 230000007613 environmental effect Effects 0.000 title claims abstract description 25
- 230000006870 function Effects 0.000 claims abstract description 77
- 238000004519 manufacturing process Methods 0.000 claims abstract description 5
- 230000004044 response Effects 0.000 claims description 38
- 210000005069 ears Anatomy 0.000 claims description 14
- 230000006835 compression Effects 0.000 claims description 9
- 238000007906 compression Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 230000009467 reduction Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 11
- 230000002829 reductive effect Effects 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 8
- 230000008878 coupling Effects 0.000 description 6
- 238000010168 coupling process Methods 0.000 description 6
- 238000005859 coupling reaction Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 5
- 238000004088 simulation Methods 0.000 description 5
- 230000005236 sound signal Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008030 elimination Effects 0.000 description 2
- 238000003379 elimination reaction Methods 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 208000032041 Hearing impaired Diseases 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000006735 deficit Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 208000016354 hearing loss disease Diseases 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1041—Mechanical or electronic switches, or control elements
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
- H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1016—Earpieces of the intra-aural type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2420/00—Details of connection covered by H04R, not provided for in its groups
- H04R2420/07—Applications of wireless loudspeakers or wireless microphones
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Otolaryngology (AREA)
- General Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Neurosurgery (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Circuit For Audible Band Transducer (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
Abstract
The invention provides a method and a device for enhancing environmental perception by hearing, which are used for acquiring original audio collected by a sound collection unit; processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of an acoustic transfer function generated by a sound production source reaching the position of the sound collection unit and the position of the ear; outputting the final audio to a receiving device; the method and the device for enhancing the environmental perception by the auditory sense can correct the difference between the original audio acquired by the sound acquisition unit and the audio supposed to be heard by the position of the human ear caused by the difference between the position of the sound acquisition unit and the position of the human ear, so that the audio sent to the receiving device can restore the real environmental sound.
Description
Technical Field
The invention relates to the technical field of auditory assistance, in particular to a method and a device for enhancing environmental perception by auditory sense.
Background
The existing auxiliary hearing products, such as auxiliary hearing earphones, hearing aids and the like, mainly collect noises and voices in the environment according to a microphone carried by the structure of the product, and then remove the noises in the collected sound source and perform voice enhancement processing by using a digital signal processing chip or a machine learning chip in the product through calculation of an algorithm; although this method can properly amplify the speech signal and remove unnecessary environmental noise, the speech intelligibility is sometimes reduced due to the impairment of speech details caused by excessive strong noise reduction processing. For a hearing-impaired user, although the volume of the voice can be amplified, the user cannot necessarily effectively understand the content of the voice because the comprehensibility of the voice is reduced, and this type of product for assisting hearing mainly aims at improving the comprehensibility of the voice when the user has hearing impairment.
Another technique is to use the microphone of the far-end mobile phone to make the microphone closer to the speaker, and transmit the audio signal of the microphone at the mobile phone end or the receiving end to the earphone or the hearing aid end in the form of wireless signal by wireless and low delay, so that the receiving end can provide the person far away from the sound-producing end, such as picking up the voice in the next room; in the function of the new apple mobile phone, a function of 'instant listening' is provided, namely a similar mode is practiced, and a microphone at the mobile phone end is used for receiving sound and transmitting the sound to a Bluetooth headset.
In the apple us patent USP 10332538Method and system for speed enhancement using a remote microphone, a speech enhancement system for a remote microphone is disclosed having a wireless receiver receiving a signal from a first microphone of a remote device. The delay buffer receives the second microphone signal from the second microphone and delays the adjustable delay. The adjustable delay is based on a difference between the wireless delay and the acoustic delay. The noise suppressor generates an output audio signal for the earpiece speaker based on the first microphone signal and the adjustably delayed second microphone signal; however, this patent mainly uses the delay difference between the remote microphone (first microphone) and the receiving end (second microphone) to superimpose the delay difference between the wireless and the acoustic signals before generating the signals, which only can compensate the time difference;
in addition, the conventional hearing aid usually only simply transmits the sound received by an external microphone to an in-ear sensor, and then the sound is processed to enhance the human voice without performing corresponding spatial sound field processing, so that the spatial perception of the real human ear hearing to the environment, such as the feeling of the sound approaching or receding, and often the noise may be amplified together.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: the method and the device for enhancing the environmental perception by the hearing can convert the perceived information into sound for enhancement, simulate the hearing environment of a real space and reduce noise aiming at the environment so as to avoid unnecessary external noise interference.
In order to solve the technical problems, the invention adopts a technical scheme that:
a method of hearing assisted perception comprising the steps of:
s1, acquiring the original audio or vibration signal acquired by the sound acquisition unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
and S3, outputting the output audio to a receiving device.
In order to solve the technical problem, the invention adopts another technical scheme as follows:
an apparatus for enhancing the perception of the environment with hearing comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the computer program implementing the steps of:
s1, acquiring the original audio or vibration signal acquired by the sound acquisition unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
and S3, outputting the output audio to a receiving device.
The invention has the beneficial effects that: the final audio frequency is obtained after the original audio frequency collected by the sound collection unit passes through the pre-generated human ear space acoustic compensation transfer function, the difference between the original audio frequency obtained by the sound collection unit and the audio frequency which should be heard at the position of the human ear caused by the difference between the position of the sound collection unit and the position of the human ear can be corrected, so that the audio transmitted to the receiving device can restore the real environmental sound, the acoustic compensation transfer function of the human ear space can be generated in advance according to the preset condition, namely, the acoustic compensation function of the human ear space is not invariable, and can be adjusted according to different specific conditions to more flexibly restore the environmental sound, because part of the vibration information belonging to low frequency is not sensed by ear, the information to be sensed is loaded on the received information by using the audio modulation technology, so as to enhance the sensing and understanding of the environmental sound in an auditory way.
Drawings
FIG. 1 is a flow chart illustrating steps of a method for enhancing the perception of the environment aurally in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of an apparatus for enhancing the perception of the environment acoustically according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a system for enhancing the perception of the environment acoustically according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating an implementation of a method for enhancing the perception of the environment aurally according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of the position of a sound point determined by human ears;
FIG. 6 is a diagram of binaural time difference and binaural volume difference versus collected sound feature difference for a speaker;
FIG. 7 is a diagram of a pulse response signal according to an embodiment of the present invention;
fig. 8 is a diagram of binaural receive pulse responses with sounding sources in different positions, in accordance with an embodiment of the present invention;
FIG. 9 is a schematic diagram of audio modulation according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of the waveforms before and after audio modulation according to an embodiment of the present invention;
description of reference numerals:
1. an apparatus for enhancing the perception of the environment aurally; 2. a processor; 3. a memory.
Detailed Description
In order to explain technical contents, achieved objects, and effects of the present invention in detail, the following description is made with reference to the accompanying drawings in combination with the embodiments.
Referring to fig. 1 and 3 to 9, a method for enhancing environmental perception with hearing includes steps;
s1, acquiring the original audio or vibration signal acquired by the sound acquisition unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
and S3, outputting the output audio to a receiving device.
From the above description, the beneficial effects of the present invention are: the original audio collected by the sound collection unit is subjected to the pre-generated human ear space acoustic compensation transfer function to obtain the original audio, the difference between the original audio obtained by the sound collection unit and the audio which should be heard by the position of the human ear caused by the difference between the position of the sound collection unit and the position of the human ear can be corrected, so that the audio sent to the receiving device can restore real environmental sound, the human ear space acoustic compensation transfer function can be pre-generated according to preset conditions, namely the human ear space acoustic compensation function is not constant, the adjustment can be made according to the difference of specific conditions, and the environmental sound can be restored more flexibly.
Further, the S2 specifically includes:
processing the original audio by a preset beam forming method to obtain a first audio;
denoising the first audio through single-channel voice to obtain a second audio;
and processing the second audio frequency through the pre-generated ear space acoustic compensation transfer function to obtain a final audio frequency, wherein the ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound production source reaches the position of the sound collection unit and the position of the ear.
According to the description, the original audio is processed through the beam forming method, the voice signals in the preset direction are reserved, the voice signals in the non-preset direction are restrained, directional noise reduction is achieved, simulation of real environment sound is enhanced, single-channel voice noise reduction is conducted on the audio subjected to beam forming, the noise elimination quality is improved, and the influence on part of the audio expected to be reserved when noise is eliminated is reduced.
Further, the process of the pre-generated ear space acoustic compensation transfer function in S2 specifically includes:
Ym(t)＝hm(t)×X(t)；
Year(t)＝hm(t)×hc(t)×X(t)；
wherein h ism(t) represents the spatial pulse response from the sound source to the sound collection unit, X (t) represents the original pulse response signal from the sound source, Ym(t) represents the sound collection unit pulse wave response signal collected by the sound collection unit; y isear(t) represents the human ear pulse wave response signal, h, collected by the human earc(t) is the human ear space acoustic compensation transfer function.
It can be known from the above description that, before the sound collection unit is first matched with a user, the ear space acoustic compensation transfer function is generated in advance through the above process, so that personalized matching service can be provided for different users, adaptability is improved, and environment restoration with the same quality of sound is provided for different users.
Further, the S2 specifically includes:
the S1 specifically includes:
acquiring a left ear original audio and a right ear original audio acquired by a sound acquisition unit;
the S2 specifically includes:
processing the original audio of the left ear through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the original audio of the right ear through a pre-generated human ear space acoustic compensation transfer function to obtain the audio of the right ear;
processing the left ear audio and the right ear audio by a sound mixer to obtain a fourth audio;
and adjusting the frequency response of the fourth audio to obtain the final audio.
It can be known from the above description that the left and right ears are distinguished when the audio is collected, the original audio of the left ear and the original audio of the right ear are obtained respectively, the original audio of the left ear and the original audio of the right ear are processed respectively by using the pre-generated acoustic compensation transfer function of the human ear space, so as to obtain the audio of the left ear and the audio of the right ear, the audio of the left ear and the audio of the right ear are mixed by the sound mixer, the sounds of the left and the right channels are fused into the dual-channel, and the frequency response adjustment is performed on the audio of the dual-channel, so that the noise reduction and the simulation of the real sound receiving.
Further, the S2 further includes:
performing dynamic compression and output gain control on the final audio to obtain an output audio;
the S3 specifically includes:
outputting the output audio to a receiving device.
According to the description, the final audio is subjected to dynamic compression and output gain, the size of the final audio is reduced through the dynamic compression, the audio output speed is convenient to output, the time delay caused by audio processing is reduced, the voice closer to the real-time situation is provided for a user, the output gain improves the output quality of the final audio, and the loss caused in the audio transmission process is reduced to a certain extent.
Referring to fig. 2, an apparatus for enhancing environmental perception with hearing comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
s1, acquiring the original audio or vibration signal acquired by the sound acquisition unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
and S3, outputting the output audio to a receiving device.
The invention has the beneficial effects that: the original audio collected by the sound collection unit is subjected to the pre-generated human ear space acoustic compensation transfer function to obtain the original audio, the difference between the original audio obtained by the sound collection unit and the audio which should be heard by the position of the human ear caused by the difference between the position of the sound collection unit and the position of the human ear can be corrected, so that the audio sent to the receiving device can restore real environmental sound, the human ear space acoustic compensation transfer function can be pre-generated according to preset conditions, namely the human ear space acoustic compensation function is not constant, the adjustment can be made according to the difference of specific conditions, and the environmental sound can be restored more flexibly.
Further, the S2 specifically includes:
processing the original audio by a preset beam forming method to obtain a first audio;
denoising the first audio through single-channel voice to obtain a second audio;
and processing the second audio frequency through the pre-generated ear space acoustic compensation transfer function to obtain a final audio frequency, wherein the ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound production source reaches the position of the sound collection unit and the position of the ear.
According to the description, the original audio is processed through the beam forming method, the voice signals in the preset direction are reserved, the voice signals in the non-preset direction are restrained, directional noise reduction is achieved, simulation of real environment sound is enhanced, single-channel voice noise reduction is conducted on the audio subjected to beam forming, the noise elimination quality is improved, and the influence on part of the audio expected to be reserved when noise is eliminated is reduced.
Further, the process of the pre-generated ear space acoustic compensation transfer function in S2 specifically includes:
Ym(t)＝hm(t)×X(t)；
Year(t)＝hm(t)×hc(t)×X(t)；
wherein h ism(t) represents the spatial pulse response from the sound source to the sound collection unit, X (t) represents the original pulse response signal from the sound source, Ym(t) represents the sound collection unit pulse wave response signal collected by the sound collection unit; y isear(t) represents the human ear pulse wave response signal, h, collected by the human earc(t) is the human ear space acoustic compensation transfer function.
It can be known from the above description that, before the sound collection unit is first matched with a user, the ear space acoustic compensation transfer function is generated in advance through the above process, so that personalized matching service can be provided for different users, adaptability is improved, and environment restoration with the same quality of sound is provided for different users.
Further, the S2 specifically includes:
the S1 specifically includes:
acquiring a left ear original audio and a right ear original audio acquired by a sound acquisition unit;
the S2 specifically includes:
processing the original audio of the left ear through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the original audio of the right ear through a pre-generated human ear space acoustic compensation transfer function to obtain the audio of the right ear;
processing the left ear audio and the right ear audio by a sound mixer to obtain a fourth audio;
and adjusting the frequency response of the fourth audio to obtain the final audio.
It can be known from the above description that the left and right ears are distinguished when the audio is collected, the original audio of the left ear and the original audio of the right ear are obtained respectively, the original audio of the left ear and the original audio of the right ear are processed respectively by using the pre-generated acoustic compensation transfer function of the human ear space, so as to obtain the audio of the left ear and the audio of the right ear, the audio of the left ear and the audio of the right ear are mixed by the sound mixer, the sounds of the left and the right channels are fused into the dual-channel, and the frequency response adjustment is performed on the audio of the dual-channel, so that the noise reduction and the simulation of the real sound receiving.
Further, the S2 further includes:
performing dynamic compression and output gain control on the final audio to obtain an output audio;
the S3 specifically includes:
outputting the output audio to a receiving device.
According to the description, the final audio is subjected to dynamic compression and output gain, the size of the final audio is reduced through the dynamic compression, the audio output speed is convenient to output, the time delay caused by audio processing is reduced, the voice closer to the real-time situation is provided for a user, the output gain improves the output quality of the final audio, and the loss caused in the audio transmission process is reduced to a certain extent.
Referring to fig. 1, fig. 3 and fig. 4, a first embodiment of the present invention is:
a method of enhancing the perception of the environment aurally, comprising the steps of: s1, acquiring the original audio collected by the sound collection unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
and S3, outputting the output audio to a receiving device.
S1, acquiring the original audio or vibration signal acquired by the Sound acquisition Unit (Sound Capture Unit);
the method specifically comprises the following steps:
acquiring a left ear original audio and a right ear original audio acquired by a sound acquisition unit;
in an alternative embodiment, the sound collection unit may be a single microphone or a microphone array, and the type of the microphone may be a directional microphone or an omnidirectional microphone;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears; coupling the vibration signal to the final audio frequency through frequency modulation to obtain an output audio frequency;
the method specifically comprises the following steps:
processing the original audio by a preset Beam Forming (Beam Forming) method to obtain a first audio; the original audio comprises a left-ear original audio and a right-ear original audio;
in an optional implementation mode, the preset beam forming method corresponds to the sound collecting unit, and is particularly suitable for setting different beam forming methods for different microphone placing modes;
subjecting the first audio to Single Channel Noise Reduction (Single Channel Noise Reduction) to obtain a second audio; the audio processed by the beam forming method is a Mono Signal (Mono Signal), and the effect of using single-channel voice noise reduction to promote voice component elements and remove environmental noise is better at the moment;
processing the original audio of the left ear through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the original audio of the right ear through a pre-generated human ear space acoustic compensation transfer function to obtain the audio of the right ear;
processing the left ear audio and the right ear audio by a Mixer (Mixer) to obtain a fourth audio; if the device is provided with two or more sound acquisition units, the device can simulate the sound receiving units of the left ear and the right ear, and the sound signals of the left sound channel and the right sound channel are fused into the sound signals of the double sound channels through the sound mixer of the double-ear hearing;
adjusting the frequency response of the fourth audio to obtain a final audio;
performing dynamic compression and output gain control on the final audio to obtain an output audio;
s3, outputting the output audio to a receiving device (Receving Unit);
referring to fig. 3, in an alternative embodiment, the receiving device (receiving Unit) may be a bluetooth Noise reduction headset (Active Noise cancellation headset);
the output audio can be output to a receiving device through wireless transmission or wired transmission;
in an alternative embodiment, a mobile phone is introduced as a transmission Unit (Transmitting Unit), and the output audio is output to a receiving device through the mobile phone;
in an alternative embodiment, the sound collection unit may be integrated on the same device as the receiving device.
Referring to fig. 5 to 8, a second embodiment of the present invention is:
human ears are mainly used for determining the direction of sound according to spatial localization of binaural hearing, and binaural hearing is mainly based on two factors, i.e. binaural Time Difference (ITD) and binaural Level Difference (ILD), when determining the direction of a speaker, please refer to fig. 6, where binaural Time Difference is the Time Difference between the arrival of a speaker's sound at the left and right ears, and binaural volume Difference is the sound volume Difference transmitted by the speaker's sound source to the left and right ears in space, and taking fig. 5 as an example, the Time Difference between the arrival of the speaker a's speech at the left ear is smaller than that at the right ear, and the arrival of the speaker B speech at the left ear is larger than that at the right ear. The calculation formula of the ITD is as follows:
ITD＝(rθ+sinθ)/s；
where r is the head radius (about 9 cm), θ is the azimuth angle, and s is the speed of sound in cm/s.
A method for enhancing the perception of the environment with hearing, which is different from the first embodiment in that:
the process of the pre-generated ear space acoustic compensation transfer function in S2 specifically includes:
Ym(t)＝hm(t)×X(t)；
Year(t)＝hm(t)×hc(t)×X(t)；
wherein h ism(t) represents the spatial pulse response of the sound source to the sound collection unit, which is transparent to Ym(t) and X (t) are obtained by separate tests, X (t) represents the original pulse response signal from the sound source, Ym(t) represents the sound collection unit pulse wave response signal collected by the sound collection unit; y isear(t) represents the human ear pulse wave response signal, h, collected by the human earc(t) is the ear space acoustic compensation transfer function;
in an alternative embodiment, Y may be collected at the sound collection unit by emitting the excitation signal at the same locationm(t), Y is collected at the earear(t), h can be obtained by only carrying out deconvolution on the twoc(t); x (t) is a known excitation signal, in particular, Year(t) and Ym(t) obtaining h by performing the mathematical operation of deconvolutionc(t) is determined after h is determinedcIn the actual quoted scene after (t), the method can be carried out according to Y collected by the sound collection unitm(t) calculating Y of human earear(t), compensation is achieved;
specifically, please refer to fig. 7, which is a schematic diagram of pulse wave response signals, fig. 8 is a schematic diagram of pulse wave response signals sent by sound sources at different positions, and differences of the pulse wave response signals received by the left ear and the right ear, when the sound sources are right in front, the magnitudes of the pulse wave response signals received by the left ear and the right ear are similar, when the sound sources are close to the right ear, the pulse wave response signal received by the left ear is smaller than that received by the right ear, and when the sound sources are in the left ear, the pulse wave response signal received by the right ear is smaller than that;
referring to fig. 10, the sound of the sea wave (broadband sound), the sound of the heartbeat (low-frequency sound, frequency lower than 20Hz), and the sound obtained by modulating and combining the sound of the sea wave and the sound of the heartbeat at the carrier frequency in step S2 (including broadband and low frequency) are respectively from top to bottom, so that the waveform characteristics of the sound of the sea wave and the sound of the heartbeat are well maintained;
in practical application, the dummy head with the same head circumference and ear position as the user can be customized, and the collection device is worn at the ear position of the dummy head to collect the pulse response (Impulse response) signal Y emitted by the sound sourceear(t) simultaneously collecting the pulse wave response signal Y emitted by the same sound source through the sound collecting unitm(t), the ear space acoustic compensation transfer function for the wearing position of the sound collection unit to which the user is accustomed can be calculated.
Referring to fig. 2 to 4, a third embodiment of the present invention is:
an apparatus 1 for enhancing the perception of a context with hearing comprising a processor 2, a memory 3 and a computer program stored on the memory 3 and executable on the processor 2, the processor 2 implementing the steps of the first or second embodiment when executing the computer program;
in this embodiment, a device for enhancing the environmental perception by hearing is an abstract concept, which may be an independent device, or a combination of several terminals to implement the steps in the first embodiment or the second embodiment; referring to fig. 3, a Signal Processing Unit (Signal Processing Unit) and a transmission device (Transmitting Unit) form an auditory auxiliary noise reduction device; referring to fig. 4, the Sound collection units (Sound Capture units) also jointly form an auxiliary hearing reduction device;
wherein, the sound collection unit can independently become another device, also can integrate on bluetooth headset.
In summary, the present invention provides a method and an apparatus for enhancing environmental perception by auditory sense, wherein the original audio collected by a sound collection unit is processed by a pre-generated acoustic compensation transfer function of human ear space, so as to compensate the difference of the acoustic transfer function between the position of the sound collection unit and the position of human ear when a sound source arrives at the sound collection unit due to the difference between the position of the sound collection unit and the position of the human ear, enhance the simulation degree of the environment by starting with details, and provide more reductive and real auditory sense assistance for a user; moreover, the voice acquired by the voice acquisition unit is processed by matching with the beam forming method, the voice capture in a specific direction can be enhanced, the direction sense of the voice is enhanced, the reality sense of the voice is improved, the single-channel voice noise reduction is carried out on the voice frequency processed by the beam forming method, the pertinence is strong, and the noise reduction effect is good; distinguish left ear and right ear when sound acquisition unit gathers, further simulate the real environment that people's ear received sound, promote the quality of the output audio frequency of final generation, and merge the audio frequency of left ear and right ear through the audio mixer, realize the dual channel output of final audio frequency, can also introduce the cell-phone and convey the receiving arrangement with the audio frequency as transmission device, the cell-phone shell carries out the conveying of audio frequency through wired and wireless mode, be suitable for multiple scene, and transmission speed is fast, do not influence the timeliness of audio frequency.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all equivalent changes made by using the contents of the present specification and the drawings, or applied directly or indirectly to the related technical fields, are included in the scope of the present invention.
Claims (10)
1. A method of enhancing the perception of the environment with hearing comprising the steps of;
s1, acquiring the original audio collected by the sound collection unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears;
and S3, outputting the final audio to a receiving device.
2. The method for enhancing the environmental perception with hearing according to claim 1, wherein the S2 is specifically:
processing the original audio by a preset beam forming method to obtain a first audio;
denoising the first audio through single-channel voice to obtain a second audio;
and processing the second audio frequency through the pre-generated ear space acoustic compensation transfer function to obtain a final audio frequency, wherein the ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound production source reaches the position of the sound collection unit and the position of the ear.
3. The method for enhancing environmental perception with hearing as claimed in claim 1, wherein the pre-generated ear space acoustic compensation transfer function in S2 is specifically:
Ym(t)＝hm(t)×X(t)；
Year(t)＝hm(t)×hc(t)×X(t)；
wherein h ism(t) represents the spatial pulse response from the sound source to the sound collection unit, X (t) represents the original pulse response signal from the sound source, Ym(t) represents the sound collection unit pulse wave response signal collected by the sound collection unit; y isear(t) represents the human ear pulse wave response signal, h, collected by the human earc(t) is the human ear space acoustic compensation transfer function.
4. The method for enhancing the environmental perception with hearing according to claim 2, wherein the S1 is specifically:
acquiring a left ear original audio and a right ear original audio acquired by a sound acquisition unit;
the S2 specifically includes:
processing the original audio of the left ear through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the original audio of the right ear through a pre-generated human ear space acoustic compensation transfer function to obtain the audio of the right ear;
processing the left ear audio and the right ear audio by a sound mixer to obtain a fourth audio;
and adjusting the frequency response of the fourth audio to obtain the final audio.
5. The method for enhancing environmental perception with hearing as recited in claim 1, wherein the S2 further comprises:
performing dynamic compression and output gain control on the final audio to obtain an output audio;
the S3 specifically includes:
outputting the output audio to a receiving device.
6. An apparatus for enhancing the perception of the environment with hearing comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the computer program performs the steps of:
s1, acquiring the original audio collected by the sound collection unit;
s2, processing the original audio through a pre-generated ear space acoustic compensation transfer function to obtain a final audio, wherein the ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collection unit and the position of the ears;
and S3, outputting the final audio to a receiving device.
7. The apparatus for enhancing the perception of the environment with hearing as claimed in claim 6, wherein the S2 is embodied as:
processing the original audio by a preset beam forming method to obtain a first audio;
denoising the first audio through single-channel voice to obtain a second audio;
and processing the second audio frequency through the pre-generated ear space acoustic compensation transfer function to obtain a final audio frequency, wherein the ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound production source reaches the position of the sound collection unit and the position of the ear.
8. The apparatus for enhancing environmental perception with hearing as claimed in claim 6, wherein the pre-generated ear space acoustic compensation transfer function in S2 is specifically:
Ym(t)＝hm(t)×X(t)；
Year(t)＝hm(t)×hc(t)×X(t)；
wherein h ism(t) represents the spatial pulse response from the sound source to the sound collection unit, X (t) represents the original pulse response signal from the sound source, Ym(t) represents the sound collection unit pulse wave response signal collected by the sound collection unit; y isear(t) represents the human ear pulse wave response signal, h, collected by the human earc(t) is the human ear space acoustic compensation transfer function.
9. The apparatus for enhancing the perception of the environment with hearing as claimed in claim 7, wherein the S1 is embodied as:
acquiring a left ear original audio and a right ear original audio acquired by a sound acquisition unit;
the S2 specifically includes:
processing the original audio of the left ear through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the original audio of the right ear through a pre-generated human ear space acoustic compensation transfer function to obtain the audio of the right ear;
processing the left ear audio and the right ear audio by a sound mixer to obtain a fourth audio;
and adjusting the frequency response of the fourth audio to obtain the final audio.
10. The apparatus for enhancing the perception of the environment with the sense of hearing as claimed in claim 6, wherein the S2 further comprises:
performing dynamic compression and output gain control on the final audio to obtain an output audio;
the S3 specifically includes:
outputting the output audio to a receiving device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239331.0A CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239331.0A CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113038322A true CN113038322A (en) | 2021-06-25 |
| CN113038322B CN113038322B (en) | 2023-08-01 |
Family
ID=76466708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110239331.0A Active CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113038322B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114067506A (en) * | 2021-12-09 | 2022-02-18 | 李科达 | Intelligent auxiliary assembly of economic Alzheimer's disease patient |
| CN114120960A (en) * | 2021-11-19 | 2022-03-01 | 武汉大学 | Auxiliary space perception system and method based on hearing |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184349A (en) * | 2007-10-10 | 2008-05-21 | 昊迪移通(北京)技术有限公司 | Three-dimensional ring sound effect technique aimed at dual-track earphone equipment |
| CN104185130A (en) * | 2013-05-23 | 2014-12-03 | Gn瑞声达A/S | Hearing aid with spatial signal enhancement |
| CN106664499A (en) * | 2014-08-13 | 2017-05-10 | 华为技术有限公司 | Audio signal processing apparatus |
| CN107071674A (en) * | 2015-10-12 | 2017-08-18 | 奥迪康有限公司 | It is configured to the hearing devices and hearing system of localization of sound source |
| CN107757537A (en) * | 2016-08-15 | 2018-03-06 | 上海汽车集团股份有限公司 | The audio control method and device of a kind of on-board engine |
| CN108769857A (en) * | 2018-06-26 | 2018-11-06 | 会听声学科技（北京）有限公司 | sound compensation method, system and earphone |
| CN208739372U (en) * | 2018-09-28 | 2019-04-12 | 北京旋极信息技术股份有限公司 | A kind of electrophone |
| CN110012406A (en) * | 2019-04-10 | 2019-07-12 | 深圳湃声声学科技有限公司 | Acoustic signal processing method, device, processor and ossiphone |
| CN110972014A (en) * | 2019-12-11 | 2020-04-07 | 歌尔智能科技有限公司 | Parameter adjustment method and device for active noise reduction earphone and wireless earphone |
| CN110996215A (en) * | 2020-02-26 | 2020-04-10 | 恒玄科技(北京)有限公司 | Method, device and computer readable medium for determining noise reduction parameters of earphone |
| CN111654806A (en) * | 2020-05-29 | 2020-09-11 | Oppo广东移动通信有限公司 | Audio playing method and device, storage medium and electronic equipment |
| CN111988690A (en) * | 2019-05-23 | 2020-11-24 | 北京小鸟听听科技有限公司 | Earphone wearing state detection method and device and earphone |
-
2021
- 2021-03-04 CN CN202110239331.0A patent/CN113038322B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184349A (en) * | 2007-10-10 | 2008-05-21 | 昊迪移通(北京)技术有限公司 | Three-dimensional ring sound effect technique aimed at dual-track earphone equipment |
| CN104185130A (en) * | 2013-05-23 | 2014-12-03 | Gn瑞声达A/S | Hearing aid with spatial signal enhancement |
| CN106664499A (en) * | 2014-08-13 | 2017-05-10 | 华为技术有限公司 | Audio signal processing apparatus |
| CN107071674A (en) * | 2015-10-12 | 2017-08-18 | 奥迪康有限公司 | It is configured to the hearing devices and hearing system of localization of sound source |
| CN107757537A (en) * | 2016-08-15 | 2018-03-06 | 上海汽车集团股份有限公司 | The audio control method and device of a kind of on-board engine |
| CN108769857A (en) * | 2018-06-26 | 2018-11-06 | 会听声学科技（北京）有限公司 | sound compensation method, system and earphone |
| CN208739372U (en) * | 2018-09-28 | 2019-04-12 | 北京旋极信息技术股份有限公司 | A kind of electrophone |
| CN110012406A (en) * | 2019-04-10 | 2019-07-12 | 深圳湃声声学科技有限公司 | Acoustic signal processing method, device, processor and ossiphone |
| CN111988690A (en) * | 2019-05-23 | 2020-11-24 | 北京小鸟听听科技有限公司 | Earphone wearing state detection method and device and earphone |
| CN110972014A (en) * | 2019-12-11 | 2020-04-07 | 歌尔智能科技有限公司 | Parameter adjustment method and device for active noise reduction earphone and wireless earphone |
| CN110996215A (en) * | 2020-02-26 | 2020-04-10 | 恒玄科技(北京)有限公司 | Method, device and computer readable medium for determining noise reduction parameters of earphone |
| CN111654806A (en) * | 2020-05-29 | 2020-09-11 | Oppo广东移动通信有限公司 | Audio playing method and device, storage medium and electronic equipment |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114120960A (en) * | 2021-11-19 | 2022-03-01 | 武汉大学 | Auxiliary space perception system and method based on hearing |
| CN114120960B (en) * | 2021-11-19 | 2024-05-03 | 武汉大学 | Auxiliary space sensing system and method based on hearing |
| CN114067506A (en) * | 2021-12-09 | 2022-02-18 | 李科达 | Intelligent auxiliary assembly of economic Alzheimer's disease patient |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113038322B (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10431239B2 (en) | Hearing system | |
| US10123134B2 (en) | Binaural hearing assistance system comprising binaural noise reduction | |
| US9930456B2 (en) | Method and apparatus for localization of streaming sources in hearing assistance system | |
| CN106937196B (en) | Hearing device | |
| CN105898651B (en) | Hearing system comprising separate microphone units for picking up the user's own voice | |
| CN105898662B (en) | Paired microphone unit and hearing system comprising the same | |
| US8391522B2 (en) | Method and system for wireless hearing assistance | |
| US8391523B2 (en) | Method and system for wireless hearing assistance | |
| EP3468228B1 (en) | Binaural hearing system with localization of sound sources | |
| US20100195836A1 (en) | Wireless communication system and method | |
| EP3883266A1 (en) | A hearing device adapted to provide an estimate of a user's own voice | |
| US20160192090A1 (en) | Method of superimposing spatial auditory cues on externally picked-up microphone signals | |
| CN113038322B (en) | Method and device for enhancing environment perception by hearing | |
| WO2019114397A1 (en) | Microphone neck ring earphone | |
| EP2916320A1 (en) | Multi-microphone method for estimation of target and noise spectral variances | |
| EP3041270B1 (en) | A method of superimposing spatial auditory cues on externally picked-up microphone signals | |
| EP4325892A1 (en) | Method of audio signal processing, hearing system and hearing device | |
| KR101391855B1 (en) | Simulator for selecting hearing aid | |
| Choi et al. | Implementation of Digital Hearing Aid Using Bluetooth Audio Digital Signal Processor | |
| CN111327980A (en) | Hearing device providing virtual sound |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |