CN113038322B - Method and device for enhancing environment perception by hearing - Google Patents
Method and device for enhancing environment perception by hearing Download PDFInfo
- Publication number
- CN113038322B CN113038322B CN202110239331.0A CN202110239331A CN113038322B CN 113038322 B CN113038322 B CN 113038322B CN 202110239331 A CN202110239331 A CN 202110239331A CN 113038322 B CN113038322 B CN 113038322B
- Authority
- CN
- China
- Prior art keywords
- audio
- ear
- sound
- transfer function
- human ear
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1041—Mechanical or electronic switches, or control elements
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
- H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/554—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired using a wireless connection, e.g. between microphone and amplifier or using Tcoils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1016—Earpieces of the intra-aural type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2420/00—Details of connection covered by H04R, not provided for in its groups
- H04R2420/07—Applications of wireless loudspeakers or wireless microphones
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Abstract
The invention provides a method and a device for enhancing environmental perception by hearing, which acquire original audio acquired by a sound acquisition unit; processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the position of the human ear; outputting the final audio to a receiving device; the method and the device for enhancing the environment perception by using the hearing sense can correct the difference between the original audio acquired by the sound acquisition unit and the audio which should be heard by the human ear caused by the difference between the position of the sound acquisition unit and the position of the human ear, so that the audio sent to the receiving device restores the real environment sound.
Description
Technical Field
The invention relates to the technical field of hearing assistance, in particular to a method and a device for enhancing environmental perception by hearing.
Background
The existing auxiliary hearing products, such as auxiliary hearing earphones, hearing aids and the like, mainly depend on microphones of the product structure to collect noise and voice in the environment, and then utilize a digital signal processing chip or a machine learning chip in the auxiliary hearing products to remove the noise in the collected sound source and perform voice enhancement processing through algorithm calculation; although the voice signal can be moderately amplified to remove unnecessary environmental noise, the voice detail is damaged due to excessive strong noise reduction treatment, so that the voice comprehensiveness is reduced. For users with hearing loss, although the volume of the voice can be amplified, the user cannot necessarily understand the content of the voice effectively due to the reduced comprehensiveness of the voice, and the type of auxiliary hearing product mainly aims at users with hearing loss, and the comprehensiveness of the voice is improved.
Another technique is to use the microphone of the far-end mobile phone to make the microphone closer to the speaking speaker, and transmit the audio signal of the microphone of the mobile phone end or the receiving end to the earphone or the hearing aid end in the form of wireless signal by wireless and low-delay mode, so that the receiving end can provide the voice for the person far from the speaking end, such as picking up the voice of the room in the partition wall; among the functions of the new type of apple mobile phone, there is a function called "listen in real time", which is a way similar to the practice, in which the sound is received by the microphone of the mobile phone end and transmitted to the bluetooth headset.
A speech enhancement system for a remote microphone is disclosed in apple us patent USP 10332538Method and system for speech enhancement using a remote microphone with a wireless receiver that receives signals from a first microphone of a remote device. The delay buffer receives the second microphone signal from the second microphone and delays the adjustable delay. The adjustable delay is based on a difference between the wireless delay and the acoustic delay. The noise suppressor generates an output audio signal for the earpiece speaker based on the first microphone signal and the adjustable delay second microphone signal; however, the patent mainly uses the delay difference between the remote microphone (the first microphone) and the receiving end (the second microphone) to superimpose the delay difference between the wireless and the acoustic signals before generating the signals, so that only the time difference can be compensated;
in addition, the conventional hearing aid product usually only simply transmits the sound received by an external microphone to an in-ear sensor directly, and the sound is enhanced by audio processing, and the corresponding spatial sound field processing is not performed, so that the spatial perception of the real human ear hearing to the environment, such as the sense of gradual approach or surrounding of the sound, cannot be simulated by the amplified sound, and the noise is possibly amplified together.
Disclosure of Invention
The technical problems to be solved by the invention are as follows: a method and apparatus for enhancing the perception of an environment with hearing are provided, which can convert the perceived message into sound and enhance it, and can simulate the hearing environment in real space and reduce noise for the environment to avoid unnecessary external noise interference.
In order to solve the technical problems, the invention adopts a technical scheme that:
a hearing-assisted sensing method comprising the steps of:
s1, acquiring an original audio or vibration signal acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
s3, outputting the output audio to a receiving device.
In order to solve the technical problems, the invention adopts another technical scheme that:
an apparatus for enhancing environmental awareness in hearing, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of, when executing the computer program:
s1, acquiring an original audio or vibration signal acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
s3, outputting the output audio to a receiving device.
The invention has the beneficial effects that: the method comprises the steps of obtaining final audio after passing an original audio acquired by a sound acquisition unit through a pre-generated human ear space acoustic compensation transfer function, correcting the difference between the original audio acquired by the sound acquisition unit and the audio which should be heard by the human ear caused by the difference between the position of the sound acquisition unit and the position of the human ear, enabling the audio sent to a receiving device to restore real environment sound, generating the human ear space acoustic compensation transfer function in advance according to preset conditions, namely, the human ear space acoustic compensation function is not invariable, adjusting according to specific conditions, and restoring the environment sound more flexibly.
Drawings
FIG. 1 is a flowchart illustrating steps of a method for enhancing environmental awareness in hearing according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an apparatus for enhancing environmental perception with hearing according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a system for enhancing environmental perception with hearing according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a method for enhancing environmental perception with hearing according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a person binaural judging the location of a sound point;
FIG. 6 is a diagram showing the difference between the binaural time difference and the binaural volume difference for the collected sound characteristics of the speaker;
FIG. 7 is a schematic diagram of a pulse response signal according to an embodiment of the present invention;
fig. 8 is a diagram showing a situation of binaural receiving pulse response when the sounding sources are at different positions according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of audio modulation according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of waveforms before and after audio modulation according to an embodiment of the present invention;
description of the reference numerals:
1. an apparatus for enhancing environmental perception with hearing; 2. a processor; 3. a memory.
Detailed Description
In order to describe the technical contents, the achieved objects and effects of the present invention in detail, the following description will be made with reference to the embodiments in conjunction with the accompanying drawings.
Referring to fig. 1 and fig. 3 to fig. 9, a method for enhancing environmental perception by hearing includes the steps;
s1, acquiring an original audio or vibration signal acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
s3, outputting the output audio to a receiving device.
From the above description, the beneficial effects of the invention are as follows: the original audio acquired by the sound acquisition unit is obtained after passing through the pre-generated human ear space acoustic compensation transfer function, so that the difference between the original audio acquired by the sound acquisition unit and the audio which should be heard by the human ear caused by the difference between the position of the sound acquisition unit and the position of the human ear can be corrected, the audio transmitted to the receiving device is enabled to restore the real environment sound, the human ear space acoustic compensation transfer function can be pre-generated according to the preset condition, namely, the human ear space acoustic compensation function is not invariable, the adjustment can be made according to the difference of specific situations, and the environment sound can be restored more flexibly.
Further, the S2 specifically is:
processing the original audio through a preset beam forming method to obtain a first audio;
noise reduction is carried out on the first audio through single-channel voice to obtain second audio;
and processing the second audio through the pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound source reaches the position of the sound collecting unit and the human ear position.
As can be seen from the above description, the original audio is processed by the beam forming method, the voice signals in the preset direction are reserved, the voice signals in the non-preset direction are suppressed, the directional noise reduction is realized, the simulation of the real environment sound is enhanced, the single-channel voice noise reduction is performed on the audio after the beam forming, the noise elimination quality is improved, and the influence on the part of the audio which is expected to be reserved when the noise is eliminated is reduced.
Further, the process of the pre-generated human ear space acoustic compensation transfer function in S2 specifically includes:
Y m (t)＝h m (t)×X(t)；
Y ear (t)＝h m (t)×h c (t)×X(t)；
wherein h is m (t) represents the spatial pulse response of the sounding source to the sound collection unit, X (t) represents the original pulse response signal from the sounding source, Y m (t) represents a sound collection unit pulse response signal collected by the sound collection unit; y is Y ear (t) represents the human ear pulse response signal, h, collected by the human ear c And (t) the acoustic compensation transfer function of the human ear space.
It can be seen from the above description that, before the sound collection unit is first matched with the user, the above process is used to generate the human ear space acoustic compensation transfer function in advance, so as to provide personalized matching service for different users, improve adaptability, and provide environment restoration of sound with the same quality for different users.
Further, the S2 specifically is:
the S1 specifically comprises the following steps:
acquiring left ear original audio and right ear original audio acquired by a sound acquisition unit;
the step S2 is specifically as follows:
processing the left ear original audio through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the right-ear original audio through a pre-generated human-ear space acoustic compensation transfer function to obtain right-ear audio;
processing the left ear audio and the right ear audio through a mixer to obtain fourth audio;
and carrying out frequency response adjustment on the fourth audio to obtain final audio.
As can be seen from the above description, when the audio is collected, the left ear and the right ear are distinguished to obtain the left ear original audio and the right ear original audio respectively, the left ear original audio and the right ear original audio are processed by using the pre-generated human ear space acoustic compensation transfer function to obtain the left ear audio and the right ear audio respectively, the left ear audio and the right ear audio are mixed by the mixer, the sound of the left channel and the sound of the right channel are fused into two channels, and the frequency response of the audio of the two channels is adjusted, so that the noise reduction and the simulation of the human ear to truly receive the sound are further realized.
Further, the step S2 further includes:
dynamically compressing the final audio and controlling the output gain to obtain output audio;
the step S3 is specifically as follows:
outputting the output audio to a receiving device.
As can be seen from the above description, the final audio is dynamically compressed and the gain is output, the size of the final audio is reduced by dynamic compression, the output is facilitated, the audio output speed can be accelerated, the time delay caused by audio processing is reduced, the voice closer to the real-time situation is provided for the user, the output quality of the final audio is improved by the output gain, and the loss caused in the audio transmission process is reduced to a certain extent.
Referring to fig. 2, an apparatus for enhancing environmental perception with hearing includes a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
s1, acquiring an original audio or vibration signal acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
s3, outputting the output audio to a receiving device.
The invention has the beneficial effects that: the original audio acquired by the sound acquisition unit is obtained after passing through the pre-generated human ear space acoustic compensation transfer function, so that the difference between the original audio acquired by the sound acquisition unit and the audio which should be heard by the human ear caused by the difference between the position of the sound acquisition unit and the position of the human ear can be corrected, the audio transmitted to the receiving device is enabled to restore the real environment sound, the human ear space acoustic compensation transfer function can be pre-generated according to the preset condition, namely, the human ear space acoustic compensation function is not invariable, the adjustment can be made according to the difference of specific situations, and the environment sound can be restored more flexibly.
Further, the S2 specifically is:
processing the original audio through a preset beam forming method to obtain a first audio;
noise reduction is carried out on the first audio through single-channel voice to obtain second audio;
and processing the second audio through the pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound source reaches the position of the sound collecting unit and the human ear position.
As can be seen from the above description, the original audio is processed by the beam forming method, the voice signals in the preset direction are reserved, the voice signals in the non-preset direction are suppressed, the directional noise reduction is realized, the simulation of the real environment sound is enhanced, the single-channel voice noise reduction is performed on the audio after the beam forming, the noise elimination quality is improved, and the influence on the part of the audio which is expected to be reserved when the noise is eliminated is reduced.
Further, the process of the pre-generated human ear space acoustic compensation transfer function in S2 specifically includes:
Y m (t)＝h m (t)×X(t)；
Y ear (t)＝h m (t)×h c (t)×X(t)；
wherein h is m (t) represents the spatial pulse response of the sounding source to the sound collection unit, X (t) represents the original pulse response signal from the sounding source, Y m (t) represents a sound collection unit pulse response signal collected by the sound collection unit; y is Y ear (t) represents the human ear pulse response signal, h, collected by the human ear c And (t) the acoustic compensation transfer function of the human ear space.
It can be seen from the above description that, before the sound collection unit is first matched with the user, the above process is used to generate the human ear space acoustic compensation transfer function in advance, so as to provide personalized matching service for different users, improve adaptability, and provide environment restoration of sound with the same quality for different users.
Further, the S2 specifically is:
the S1 specifically comprises the following steps:
acquiring left ear original audio and right ear original audio acquired by a sound acquisition unit;
the step S2 is specifically as follows:
processing the left ear original audio through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the right-ear original audio through a pre-generated human-ear space acoustic compensation transfer function to obtain right-ear audio;
processing the left ear audio and the right ear audio through a mixer to obtain fourth audio;
and carrying out frequency response adjustment on the fourth audio to obtain final audio.
As can be seen from the above description, when the audio is collected, the left ear and the right ear are distinguished to obtain the left ear original audio and the right ear original audio respectively, the left ear original audio and the right ear original audio are processed by using the pre-generated human ear space acoustic compensation transfer function to obtain the left ear audio and the right ear audio respectively, the left ear audio and the right ear audio are mixed by the mixer, the sound of the left channel and the sound of the right channel are fused into two channels, and the frequency response of the audio of the two channels is adjusted, so that the noise reduction and the simulation of the human ear to truly receive the sound are further realized.
Further, the step S2 further includes:
dynamically compressing the final audio and controlling the output gain to obtain output audio;
the step S3 is specifically as follows:
outputting the output audio to a receiving device.
As can be seen from the above description, the final audio is dynamically compressed and the gain is output, the size of the final audio is reduced by dynamic compression, the output is facilitated, the audio output speed can be accelerated, the time delay caused by audio processing is reduced, the voice closer to the real-time situation is provided for the user, the output quality of the final audio is improved by the output gain, and the loss caused in the audio transmission process is reduced to a certain extent.
Referring to fig. 1, 3 and 4, a first embodiment of the present invention is as follows:
a method for enhancing environmental perception in hearing, comprising the steps of: s1, acquiring original audio acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
s3, outputting the output audio to a receiving device.
S1, acquiring an original audio or vibration signal acquired by a sound acquisition unit (Sound Capture Unit);
the method comprises the following steps:
acquiring left ear original audio and right ear original audio acquired by a sound acquisition unit;
in an alternative embodiment, the sound collection unit may be a single microphone or an array of microphones, the type of microphone may be a directional microphone or an omni-directional microphone;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position; coupling the vibration signal to the final audio through frequency modulation to obtain output audio;
the method comprises the following steps:
processing the original audio through a preset Beam Forming (Beam Forming) method to obtain a first audio; the original audio package is left ear original audio and right ear original audio;
in an alternative embodiment, the preset beam forming method corresponds to the sound collecting unit, and is particularly suitable for setting different beam forming methods for different microphone placement modes;
-noise reducing (Single Channel Noise Reduction) the first audio by single channel speech to obtain a second audio; the audio processed by the beam forming method is a Mono Signal, and the effect of improving the voice component elements and removing the environmental noise by using single-channel voice noise reduction is good;
processing the left ear original audio through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the right-ear original audio through a pre-generated human-ear space acoustic compensation transfer function to obtain right-ear audio;
processing the left ear audio and the right ear audio through a Mixer to obtain fourth audio; if the two or more sound collecting units are provided, the sound receiving units of the left ear and the right ear can be simulated, and the sound signals of the left channel and the sound signals of the right channel are fused into a double-channel sound signal through a double-ear hearing mixer;
performing frequency response adjustment on the fourth audio to obtain final audio;
dynamically compressing the final audio and controlling the output gain to obtain output audio;
s3, outputting the output audio to a receiving device (receiving Unit);
referring to fig. 3, in an alternative embodiment, the receiving device (receiving Unit) may be a bluetooth noise reduction headset (Active Noise Canceling Headest);
the output audio can be output to the receiving device in a wireless transmission or wired transmission mode;
in an alternative embodiment, a mobile phone is introduced as the transmission means (Transmitting Unit), and the output audio is output to the receiving means through the mobile phone;
in an alternative embodiment, the sound collection unit may be integrated on the same device as the receiving device.
Referring to fig. 5 to 8, a second embodiment of the present invention is as follows:
the direction of sound is determined by human ears mainly according to spatial localization of binaural hearing, and binaural hearing is determined by mainly according to two factors, namely binaural time difference (Interaural Time Difference, ITD) and binaural volume difference (Interaural Level Difference, ILD), please refer to fig. 6, wherein binaural time difference is a time difference of sound of a speaker reaching left and right ears, binaural volume difference refers to a sound volume difference of sound source of the speaker transmitted to the left and right ears from space, and taking fig. 5 as an example, a time difference of sound source of speaker a reaching left ear is smaller than that of sound source of speaker B reaching left ear, and a time of sound source of speaker B reaching left ear is larger than that of speaker B reaching right ear. The calculation formula of the ITD is as follows:
ITD＝(rθ+sinθ)/s；
where r is the head radius (about 9 cm), θ is the azimuth, and s is the speed of sound in cm/s.
A method of enhancing environmental perception with hearing, which differs from embodiment one in that:
the process of the pre-generated human ear space acoustic compensation transfer function in S2 specifically includes:
Y m (t)＝h m (t)×X(t)；
Y ear (t)＝h m (t)×h c (t)×X(t)；
wherein h is m (t) represents the spatial pulse response of the sound source to the sound collection unit, which can be transmitted through Y m (t) and X (t) are obtained by separate tests, X (t) represents the original pulse response signal emitted by the sounding source, Y m (t) represents a sound collection unit pulse response signal collected by the sound collection unit; y is Y ear (t) represents the human ear pulse response signal, h, collected by the human ear c (t) is the acoustic compensation transfer function of the human ear space;
in an alternative embodiment, Y can be acquired at the sound acquisition unit by sending out excitation signals at the same location m (t) acquisition of Y at the ear ear (t) deconvoluting the two to obtain h c (t); x (t) is a known excitation signal, specifically Y ear (t) and Y m (t) performing inverse convolution mathematical operation to obtain h c The value of (t) in determining h c In the actual cited scene after (t), according to the Y collected by the sound collecting unit m (t) calculating Y at the human ear ear (t) effecting compensation;
specifically, referring to fig. 7, a schematic diagram of pulse response signals is shown, fig. 8 shows that sound sources at different positions emit pulse response signals, the positions of the left and right ears receive the difference of the pulse response signals, the sizes of the pulse response signals received by the positions of the left and right ears are similar when the sound source is in front of the sound source, the pulse response signals received by the left ear are smaller than those received by the right ear when the sound source is close to the right ear, and the pulse response signals received by the right ear are smaller than those received by the left ear when the sound source is in the left ear;
referring to fig. 10, from top to bottom, the wave sound (broadband sound), the heartbeat sound (low-frequency sound, frequency lower than 20 Hz) and the sound (including broadband and low-frequency sound) obtained by performing carrier frequency modulation and combination on the wave sound and the heartbeat sound in step S2 can well maintain the waveform characteristics of the wave sound and the heartbeat sound;
in practical application, the fake head with the same head circumference and ear position as those of the user can be customized, and the acquisition device is worn at the ear position of the fake head to acquire soundingPulse response (Impulse response) signal Y from source ear (t) and simultaneously collecting pulse response signals Y emitted by the same sound source through the sound collecting unit m (t) the human ear space acoustic compensation transfer function for the wearing position of the sound collection unit to which the user and his habit are directed can be calculated.
Referring to fig. 2 to 4, a third embodiment of the present invention is as follows:
a device 1 for enhancing environmental perception in hearing, comprising a processor 2, a memory 3 and a computer program stored on the memory 3 and executable on the processor 2, the processor 2 implementing the steps of either embodiment one or embodiment two when executing the computer program;
in this embodiment, a device for enhancing the perception of the environment by hearing is an abstract concept, which may be an independent device, or may be a combination of several ends to jointly implement each step in the first embodiment or the second embodiment; referring to fig. 3, the signal processing unit (Signal Processing Unit) and the transmission device (Transmitting Unit) form an hearing auxiliary noise reduction device; referring to fig. 4, the sound collection units (Sound Capture Unit) also together form an hearing aid noise reduction device;
the sound collecting unit can be independently used as another device or integrated on the Bluetooth headset.
In summary, the invention provides a method and a device for enhancing environmental perception by hearing, which processes the original audio acquired by a sound acquisition unit through a pre-generated human ear space acoustic compensation transfer function, compensates the acoustic transfer function difference generated when a sound source arrives at the sound acquisition unit position and the human ear position due to the difference between the sound acquisition unit position and the human ear reception position, starts from details, enhances the simulation degree of the environment, and provides more restored and real hearing assistance for users; in addition, the sound acquired by the sound acquisition unit is processed by matching with the beam forming method, so that the sound capture in a specific direction can be enhanced, the direction sense of the sound is enhanced, the sense of reality of the sound is improved, and the voice frequency processed by the beam forming method is subjected to single-channel voice noise reduction, so that the pertinence is strong, and the noise reduction effect is good; the voice receiving device has the advantages that the left ear and the right ear are distinguished when the voice collecting unit collects voice, the real environment of the voice received by the human ear is further simulated, the quality of output voice generated finally is improved, the voice of the left ear and the voice of the right ear are fused through the voice mixer, the binaural output of the final voice is achieved, a mobile phone can be introduced to serve as a transmission device to transmit the voice to the receiving device, the mobile phone shell transmits the voice in a wired and wireless mode, multiple scenes are applicable, the transmission speed is high, and timeliness of the voice is not affected.
The foregoing description is only illustrative of the present invention and is not intended to limit the scope of the invention, and all equivalent changes made by the specification and drawings of the present invention, or direct or indirect application in the relevant art, are included in the scope of the present invention.
Claims (6)
1. A method for enhancing environmental perception in hearing, comprising the steps of;
s1, acquiring original audio acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position;
s3, outputting the final audio to a receiving device;
the process of the pre-generated human ear space acoustic compensation transfer function in S2 specifically includes:
；
；
wherein, the liquid crystal display device comprises a liquid crystal display device,representing hairSpatial pulse response of sound source to said sound collection unit,/->Representing the original pulse response signal from the sounding source, < + >>Representing the pulse wave response signal of the sound collecting unit collected by the sound collecting unit;representing the human ear pulse response signal, < > acquired by the human ear>The acoustic compensation transfer function of the human ear space is obtained;
by sending out the excitation signal X (t) at the same position, the sound is collected by the sound collecting unitAt the ear collect，/>And->The mathematical operation of the inverse convolution is performed to obtain +.>Is to determine ∈>In the actual application scene, the method can be carried out according to the +.>Calculating to obtain +.>Compensation is realized;
the step S2 is specifically as follows:
processing the original audio through a preset beam forming method to obtain a first audio;
noise reduction is carried out on the first audio through single-channel voice to obtain second audio;
and processing the second audio through the pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound source reaches the position of the sound collecting unit and the human ear position.
2. A method for enhancing environmental perception in hearing according to claim 1, wherein S1 is specifically:
acquiring left ear original audio and right ear original audio acquired by a sound acquisition unit;
the step S2 is specifically as follows:
processing the left ear original audio through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the right-ear original audio through a pre-generated human-ear space acoustic compensation transfer function to obtain right-ear audio;
processing the left ear audio and the right ear audio through a mixer to obtain fourth audio;
and carrying out frequency response adjustment on the fourth audio to obtain final audio.
3. A method for enhancing environmental awareness in hearing according to claim 1, wherein S2 further comprises:
dynamically compressing the final audio and controlling the output gain to obtain output audio;
the step S3 is specifically as follows:
outputting the output audio to a receiving device.
4. An apparatus for enhancing environmental awareness in hearing, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program when executed by the processor performs the steps of:
s1, acquiring original audio acquired by a sound acquisition unit;
s2, processing the original audio through a pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of acoustic transfer functions generated when a sound source reaches the position of the sound collecting unit and the human ear position;
s3, outputting the final audio to a receiving device;
the process of the pre-generated human ear space acoustic compensation transfer function in S2 specifically includes:
；
；
wherein, the liquid crystal display device comprises a liquid crystal display device,representing the spatial pulse response of a sound source to said sound collection unit,/>Representing the original pulse response signal from the sounding source, < + >>Representing the pulse wave response signal of the sound collecting unit collected by the sound collecting unit;representing the human ear pulse response signal, < > acquired by the human ear>The acoustic compensation transfer function of the human ear space is obtained;
by sending out the excitation signal X (t) at the same position, the sound is collected by the sound collecting unitAt the ear collect，/>And->The mathematical operation of the inverse convolution is performed to obtain +.>Is to determine ∈>In the actual application scene, the method can be carried out according to the +.>Calculating to obtain +.>Compensation is realized;
the step S2 is specifically as follows:
processing the original audio through a preset beam forming method to obtain a first audio;
noise reduction is carried out on the first audio through single-channel voice to obtain second audio;
and processing the second audio through the pre-generated human ear space acoustic compensation transfer function to obtain final audio, wherein the human ear space acoustic compensation transfer function is used for compensating the difference of the acoustic transfer function generated when the sound source reaches the position of the sound collecting unit and the human ear position.
5. The device for enhancing environmental perception by hearing according to claim 4, wherein S1 is specifically:
acquiring left ear original audio and right ear original audio acquired by a sound acquisition unit;
the step S2 is specifically as follows:
processing the left ear original audio through a pre-generated human ear space acoustic compensation transfer function to obtain left ear audio;
processing the right-ear original audio through a pre-generated human-ear space acoustic compensation transfer function to obtain right-ear audio;
processing the left ear audio and the right ear audio through a mixer to obtain fourth audio;
and carrying out frequency response adjustment on the fourth audio to obtain final audio.
6. The apparatus for hearing enhanced environmental awareness of claim 4 wherein S2 further comprises:
dynamically compressing the final audio and controlling the output gain to obtain output audio;
the step S3 is specifically as follows:
outputting the output audio to a receiving device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239331.0A CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110239331.0A CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113038322A CN113038322A (en) | 2021-06-25 |
| CN113038322B true CN113038322B (en) | 2023-08-01 |
Family
ID=76466708
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110239331.0A Active CN113038322B (en) | 2021-03-04 | 2021-03-04 | Method and device for enhancing environment perception by hearing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113038322B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114067506A (en) * | 2021-12-09 | 2022-02-18 | 李科达 | Intelligent auxiliary assembly of economic Alzheimer's disease patient |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107757537A (en) * | 2016-08-15 | 2018-03-06 | 上海汽车集团股份有限公司 | The audio control method and device of a kind of on-board engine |
| CN110972014A (en) * | 2019-12-11 | 2020-04-07 | 歌尔智能科技有限公司 | Parameter adjustment method and device for active noise reduction earphone and wireless earphone |
| CN110996215A (en) * | 2020-02-26 | 2020-04-10 | 恒玄科技(北京)有限公司 | Method, device and computer readable medium for determining noise reduction parameters of earphone |
| CN111988690A (en) * | 2019-05-23 | 2020-11-24 | 北京小鸟听听科技有限公司 | Earphone wearing state detection method and device and earphone |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101184349A (en) * | 2007-10-10 | 2008-05-21 | 昊迪移通(北京)技术有限公司 | Three-dimensional ring sound effect technique aimed at dual-track earphone equipment |
| US10425747B2 (en) * | 2013-05-23 | 2019-09-24 | Gn Hearing A/S | Hearing aid with spatial signal enhancement |
| EP3132617B1 (en) * | 2014-08-13 | 2018-10-17 | Huawei Technologies Co. Ltd. | An audio signal processing apparatus |
| US9980055B2 (en) * | 2015-10-12 | 2018-05-22 | Oticon A/S | Hearing device and a hearing system configured to localize a sound source |
| CN108769857A (en) * | 2018-06-26 | 2018-11-06 | 会听声学科技（北京）有限公司 | sound compensation method, system and earphone |
| CN208739372U (en) * | 2018-09-28 | 2019-04-12 | 北京旋极信息技术股份有限公司 | A kind of electrophone |
| CN110012406A (en) * | 2019-04-10 | 2019-07-12 | 深圳湃声声学科技有限公司 | Acoustic signal processing method, device, processor and ossiphone |
| CN111654806B (en) * | 2020-05-29 | 2022-01-07 | Oppo广东移动通信有限公司 | Audio playing method and device, storage medium and electronic equipment |
-
2021
- 2021-03-04 CN CN202110239331.0A patent/CN113038322B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107757537A (en) * | 2016-08-15 | 2018-03-06 | 上海汽车集团股份有限公司 | The audio control method and device of a kind of on-board engine |
| CN111988690A (en) * | 2019-05-23 | 2020-11-24 | 北京小鸟听听科技有限公司 | Earphone wearing state detection method and device and earphone |
| CN110972014A (en) * | 2019-12-11 | 2020-04-07 | 歌尔智能科技有限公司 | Parameter adjustment method and device for active noise reduction earphone and wireless earphone |
| CN110996215A (en) * | 2020-02-26 | 2020-04-10 | 恒玄科技(北京)有限公司 | Method, device and computer readable medium for determining noise reduction parameters of earphone |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113038322A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10431239B2 (en) | Hearing system | |
| US9930456B2 (en) | Method and apparatus for localization of streaming sources in hearing assistance system | |
| CN104980865B (en) | Binaural hearing aid system including binaural noise reduction | |
| JP5315506B2 (en) | Method and system for bone conduction sound propagation | |
| US10567889B2 (en) | Binaural hearing system and method | |
| US10057693B2 (en) | Method for predicting the intelligibility of noisy and/or enhanced speech and a binaural hearing system | |
| EP3833041B1 (en) | Earphone signal processing method and system, and earphone | |
| US20100002886A1 (en) | Hearing system and method implementing binaural noise reduction preserving interaural transfer functions | |
| EP3468228B1 (en) | Binaural hearing system with localization of sound sources | |
| US9699574B2 (en) | Method of superimposing spatial auditory cues on externally picked-up microphone signals | |
| EP3883266A1 (en) | A hearing device adapted to provide an estimate of a user's own voice | |
| CN109218948B (en) | Hearing aid system, system signal processing unit and method for generating an enhanced electrical audio signal | |
| CN105744455B (en) | Method for superimposing spatial auditory cues on externally picked-up microphone signals | |
| CN113038322B (en) | Method and device for enhancing environment perception by hearing | |
| EP2916320A1 (en) | Multi-microphone method for estimation of target and noise spectral variances | |
| JP2018113681A (en) | Audition apparatus having adaptive audibility orientation for both ears and related method | |
| CN111781555B (en) | Active noise reduction earphone sound source positioning method and device with correction function | |
| EP3041270B1 (en) | A method of superimposing spatial auditory cues on externally picked-up microphone signals | |
| EP4325892A1 (en) | Method of audio signal processing, hearing system and hearing device | |
| EP4094685A1 (en) | Spectro-temporal modulation detection test unit | |
| Choi et al. | Implementation of Digital Hearing Aid Using Bluetooth Audio Digital Signal Processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |