WO2021258913A1 - 回声消除装置及方法、拾音装置及方法、终端 - Google Patents

回声消除装置及方法、拾音装置及方法、终端 Download PDF

Info

Publication number
WO2021258913A1
WO2021258913A1 PCT/CN2021/094175 CN2021094175W WO2021258913A1 WO 2021258913 A1 WO2021258913 A1 WO 2021258913A1 CN 2021094175 W CN2021094175 W CN 2021094175W WO 2021258913 A1 WO2021258913 A1 WO 2021258913A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
vibration
unit
pickup unit
terminal
Prior art date
Application number
PCT/CN2021/094175
Other languages
English (en)
French (fr)
Inventor
唐冲
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2021258913A1 publication Critical patent/WO2021258913A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M9/00—Arrangements for interconnection not involving centralised switching
    • H04M9/08—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
    • H04M9/082—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic using echo cancellers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M9/00—Arrangements for interconnection not involving centralised switching
    • H04M9/08—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic

Definitions

  • the embodiments of the present invention relate to the field of audio processing, and in particular, to an echo cancellation device and method, a sound pickup device and method, and a terminal.
  • the screen sound technology uses a screen sound device to drive the front screen and structure.
  • the screen is used as a vibrating body, and sound waves are generated by vibration.
  • the principle is to directly transmit the vibrational mechanical energy to the screen, and then let the screen replace the traditional speaker diaphragm to produce sound .
  • the entire structure of the terminal will vibrate under the action of the screen sound device. This vibration will generate a certain amount of original noise.
  • the sound pickup unit of the terminal such as a microphone, will In this process, the original noise generated by the above vibration is picked up. Because the original noise lacks a reference signal and is a non-linear noise, it cannot be directly eliminated by the noise reduction module and echo module in the related technology, which will cause the terminal screen to sound in the process echo.
  • the embodiments of the present invention provide an echo cancellation device and method, a sound pickup device and method, and a terminal, so as to at least solve the problem that the noise generated by the vibration of the terminal cannot be effectively eliminated in the process of screen sounding in the related art.
  • an echo cancellation device which is provided in a terminal, the terminal includes at least a screen sounding unit and a sound pickup unit, and the device includes:
  • a detection unit configured to detect the vibration parameter of the screen sounding unit to obtain a vibration signal, and generate a first reference signal according to the vibration signal;
  • the cancellation unit is configured to perform echo cancellation on the first echo signal in the first audio signal according to the first reference signal; wherein, the first audio signal is an audio signal picked up by the sound pickup unit, and the first The audio signal includes at least a first echo signal, and the first echo signal is formed by the noise generated by the screen sound unit driving the terminal to vibrate.
  • a sound pickup device which is provided in a terminal, the terminal includes at least a screen sounding unit, a first sound pickup unit, and a second sound pickup unit, and the device includes:
  • the detection unit further includes:
  • the first detection subunit is set within a preset range of the first pickup unit, and the first detection subunit is configured to detect the vibration parameter of the area where the first pickup unit is located in the terminal to obtain The first vibration signal;
  • the second detection subunit is arranged within the preset range of the second pickup unit, and the second detection subunit is configured to detect the vibration parameter of the area where the second pickup unit is located in the terminal to obtain Second vibration signal;
  • the device also includes:
  • the processing unit is configured to select the first sound pickup unit or the second sound pickup unit to input target audio according to the first vibration signal and the second vibration signal.
  • a terminal including the echo cancellation device and/or the sound pickup device described in the foregoing embodiment.
  • an echo cancellation method which is applied to a terminal, the terminal includes at least a screen sounding unit and a sound pickup unit, and the method includes:
  • the first audio signal is an audio signal picked up by the pickup unit, and the first audio signal includes at least A first echo signal, where the first echo signal is formed by noise generated by the screen sound unit driving the terminal to vibrate.
  • a sound pickup method which is applied to a terminal, the terminal includes at least a screen sounding unit, a first sound pickup unit, and a second sound pickup unit, and the method further includes:
  • the first sound pickup unit or the second sound pickup unit is selected to input the target audio.
  • a computer-readable storage medium in which a computer program is stored, wherein the computer program is configured to execute any one of the above methods when running Steps in the embodiment.
  • an electronic device including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above Steps in the method embodiment.
  • the vibration parameter of the screen sounding unit can be detected by the detection unit to obtain the vibration signal, and the first reference signal can be generated according to the vibration signal;
  • the first echo signal in the first audio signal performs echo cancellation; the above-mentioned first echo signal is formed by the noise generated by the vibration of the terminal driven by the screen sound unit. Therefore, the embodiment of the present invention can solve the problem that the noise generated by the terminal vibration during the screen sounding process cannot be effectively eliminated in the related art, so as to eliminate the noise generated by the terminal vibration during the screen sounding process, and improve the sound quality of the terminal input audio.
  • Fig. 1 is a schematic diagram of an application scenario of an echo cancellation device according to an embodiment of the present invention
  • Figure 2 is a block diagram of the hardware structure of a terminal according to an embodiment of the present invention.
  • Fig. 3 is a schematic diagram (1) of sound on the terminal screen according to an embodiment of the present invention.
  • Fig. 4 is a schematic diagram (2) of sound on the terminal screen according to an embodiment of the present invention.
  • Fig. 5 is a functional schematic diagram (1) of an echo cancellation device according to an embodiment of the present invention.
  • Fig. 6 is a functional schematic diagram (2) of an echo cancellation device according to an embodiment of the present invention.
  • Fig. 7 is a working schematic diagram of a cancellation unit provided according to an embodiment of the present invention.
  • Fig. 8 is a functional schematic diagram (3) of an echo cancellation device according to an embodiment of the present invention.
  • FIG. 9 is a working schematic diagram of a second reference signal according to an embodiment of the present invention.
  • Figure 10 is a working schematic diagram of an echo cancellation device according to an embodiment of the present invention.
  • FIG. 11 is a schematic diagram of echo cancellation performed by a cancellation unit according to an embodiment of the present invention.
  • Fig. 12 is a schematic diagram (3) of sound on the terminal screen according to an embodiment of the present invention.
  • Fig. 13 is a schematic diagram (4) of sound on the terminal screen according to an embodiment of the present invention.
  • Fig. 14 is a functional schematic diagram of a sound pickup device according to an embodiment of the present invention.
  • FIG. 15 is a schematic diagram (5) of sound on the terminal screen according to an embodiment of the present invention.
  • Figure 16 is a flowchart of an echo cancellation method according to an embodiment of the present invention.
  • Fig. 17 is a flowchart of a sound pickup method according to an embodiment of the present invention.
  • the following describes the application scenarios of the echo cancellation device and method, the sound pickup device and method, and the terminal in the embodiment of the present invention:
  • FIG. 1 is a schematic diagram of an application scenario of an echo cancellation device according to an embodiment of the present invention.
  • a user A uses a terminal to make a voice call with a user B.
  • the vibration generated by the device, etc. to drive the screen to vibrate and sound the terminal.
  • the terminal receives the voice message 1 sent by user B, and uses the screen sound unit to vibrate to drive the screen to vibrate and output the voice message 1 for user A to listen to; at the same time, user A uses the terminal’s pickup unit, such as a microphone, to input the voice message 2.
  • pickup unit such as a microphone
  • the vibration generated by the screen sound unit can drive the screen to vibrate and sound on the one hand, and on the other hand, it will cause the terminal to vibrate as a whole, thereby forming a certain amplitude of vibration noise. Therefore, in the process of user A inputting voice message 2 through the sound pickup unit of the terminal, in addition to the voice message 2 input by user A, the sound pickup unit of the terminal will also pick up the above-mentioned vibration and noise, and then send it to user B. Therefore, the voice message picked up by the sound pickup unit of the terminal and sent to user B is actually audio information that carries vibration and noise. For user B, since the received audio information contains vibration noise, it will greatly affect his listening effect.
  • FIG. 2 is a hardware structure block diagram of a terminal provided according to an embodiment of the present invention.
  • the mobile terminal may include one or more (only one is shown in FIG. 2) processor 002 (The processor 002 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 004 for storing data.
  • the above-mentioned mobile terminal may also include a transmission device 006 for communication functions. And input and output equipment 008.
  • FIG. 2 is only for illustration, and does not limit the structure of the above-mentioned mobile terminal.
  • the mobile terminal may also include more or fewer components than shown in FIG. 2 or have a different configuration from that shown in FIG. 1.
  • the memory 004 can be used to store computer programs, for example, software programs and modules of application software, such as the computer programs corresponding to the echo cancellation method and the sound pickup method in the embodiment of the present invention.
  • the processor 002 runs the computer program stored in the memory 004 , So as to perform various functional applications and data processing, that is, to achieve the above-mentioned methods.
  • the memory 004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 004 may further include a memory remotely provided with respect to the processor 002, and these remote memories may be connected to the mobile terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
  • the transmission device 006 is used to receive or send data via a network.
  • the above-mentioned specific examples of the network may include a wireless network provided by a communication provider of a mobile terminal.
  • the transmission device 006 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station to communicate with the Internet.
  • the transmission device 006 may be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet in a wireless manner.
  • RF Radio Frequency
  • the embodiments of the present invention provide an echo cancellation device and method, a sound pickup device and method, and a terminal, so as to effectively eliminate the noise generated by the vibration of the terminal during the screen sounding process, and improve the sound quality of the terminal input audio.
  • the following describes the echo cancellation device and method, the sound pickup device and method, and the terminal in the embodiments of the present invention:
  • the embodiment of the present invention provides an echo cancellation device, which is set in a terminal, and the terminal includes at least a screen sounding unit 106 and a sound pickup unit 108.
  • Figure 3 is a schematic diagram (1) of terminal screen sounding provided according to an embodiment of the present invention
  • Figure 4 is a schematic diagram (2) of terminal screen sounding provided according to an embodiment of the present invention, as shown in Figures 3 and 4, the implementation of the present invention
  • the terminal in the example vibrates through the screen sound unit to drive the screen to vibrate and produce sound.
  • Fig. 5 is a functional schematic diagram (1) of an echo cancellation device according to an embodiment of the present invention. As shown in Fig. 5, the device includes:
  • the detection unit 102 is configured to detect vibration parameters of the screen sounding unit to obtain a vibration signal, and generate a first reference signal according to the vibration signal;
  • the cancellation unit 104 is configured to perform echo cancellation on the first echo signal in the first audio signal according to the first reference signal; wherein the first audio signal is the audio signal picked up by the pickup unit, and the first audio signal includes at least the first The echo signal, the first echo signal is formed by the noise generated by the screen sound unit driving the terminal to vibrate.
  • the above-mentioned screen sound generating unit may be a cantilever piezoelectric ceramic or an exciter, and the above-mentioned sound pickup unit may be a microphone.
  • the vibration parameter of the screen sounding unit is used to instruct the screen sounding unit to generate vibration for outputting audio information to drive the screen to vibrate. Etc., the present invention does not limit this.
  • FIG. 6 is a functional schematic diagram (2) of the echo cancellation device provided according to an embodiment of the present invention.
  • the detection unit is 102 includes:
  • the vibration sensing subunit 1022 is arranged within a preset range of the screen sounding unit, and the vibration sensing subunit is configured to detect vibration parameters of the screen sounding unit to obtain a vibration signal;
  • the analog-to-digital conversion (ADC) subunit 1024 is configured to perform analog-to-digital conversion processing on the vibration signal to obtain the first reference signal.
  • the above-mentioned vibration sensing subunit may be constituted by a vibration sensor.
  • the vibration sensor may be an inductive vibration sensor, which can achieve the screen
  • the detection of the vibration parameters of the sounding unit further converts the mechanical vibration parameters during the vibration of the screen sounding unit into an electrical parameter signal, that is, the above-mentioned vibration signal. Since the vibration signal acquired by the vibration sensing sub-unit is an analog signal, it is necessary to perform analog-to-digital conversion processing on the vibration through the ADC sub-unit to obtain the first reference signal.
  • the vibration sensing subunit needs to be set within the preset range of the screen sounding unit.
  • the vibration sensing subunit is arranged at the side end of the screen sounding unit, and is arranged next to the screen sounding unit.
  • the vibration of the screen sounding unit in addition to driving the screen to vibrate and produce sound during the vibration of the screen sounding unit, the vibration of the screen sounding unit will also cause the entire terminal to vibrate, thereby causing vibration noise during the vibration of the terminal.
  • This vibration noise is the above-mentioned first vibration noise.
  • An echo message In the embodiment of the present invention, the above-mentioned first echo information may be picked up by the sound pickup unit, and then it may be eliminated by the above-mentioned elimination unit.
  • the above-mentioned elimination unit 104 is configured to:
  • the first audio signal and the first filtered signal are superimposed to perform echo cancellation on the first echo signal.
  • the above-mentioned inversion processing is performed on the first reference signal, that is, a signal with the opposite phase to the first reference signal is obtained, and this signal is the first filtered signal; by combining the first filtered signal with the first echo The first audio signal of the signal is superimposed, and the first filtered signal and the first echo signal can be cancelled to eliminate the first echo signal in the first audio signal.
  • the cancellation unit may be composed of a digital signal processing (Digital Signal Process, DSP) chip for audio processing, and the DSP chip performs DSP processing on the first audio signal, that is, the above-mentioned inverting the first reference signal Processing to generate a first filtered signal; and superimposing the first audio signal with the first filtered signal to perform echo cancellation on the first echo signal.
  • DSP Digital Signal Process
  • the cancellation unit in the embodiment of the present invention may be composed of an independent DSP chip, or may be integrated in a DSP chip or other processor used for audio processing of the terminal itself, which is not limited in the present invention.
  • Fig. 7 is a working schematic diagram of the cancellation unit provided according to an embodiment of the present invention.
  • the vibration sensor in the detection unit detects the vibration signal of the screen sound unit, and performs analog-to-digital conversion processing through ADC to obtain the first
  • the elimination unit can perform inversion processing on the above-mentioned first reference signal to generate a first filtered signal; the waveforms of the first reference signal and the first filtered signal are as shown in FIG. The phase of the filtered signal is opposite.
  • the foregoing first audio signal includes: a first echo signal and a second echo signal;
  • the second echo signal is formed by the sound output by the screen sound unit driving the terminal to vibrate according to the second audio signal, and the second audio signal is an audio signal pre-output by the terminal.
  • the second audio information may be a voice signal sent to the terminal by other users, or audio/video information provided by the service provider that the terminal is ready to play, that is, the audio that the terminal is ready to output/play That is the second audio information.
  • the terminal can control the screen sound unit to vibrate according to the second audio signal, so as to drive the screen to vibrate accordingly to output a desired sound.
  • the sound output by the screen can be listened to by the user on the one hand, but also picked up by the sound pickup unit of the terminal. Especially in indoor or narrow environments, the sound output by the screen is picked up by the sound pickup unit of the terminal.
  • the process can be similar to the echo transmission process.
  • the sound signal picked up by the sound pickup unit due to the sound output by the screen sound unit driving the terminal vibration is returned, which is the second echo signal in the embodiment of the present invention.
  • the first audio signal includes a target audio signal in addition to the first echo signal and the second echo signal.
  • the target audio signal is the target audio input through the pickup unit, that is, the user passes through The target audio actively input by the pickup unit.
  • the target audio is a voice message sent by the user to a third party through the terminal; in another example, the user records through the terminal
  • the target audio is the live sound recorded by the user through the terminal.
  • the sound pickup unit of the above terminal picks up the target audio signal
  • the actually picked up audio signal carries the first echo signal and the second echo signal. That is, in the above optional embodiment, the first echo signal and the second echo signal are included.
  • the second echo signal the first audio signal of the target audio signal.
  • FIG. 8 is a functional schematic diagram (3) of an echo cancellation device provided according to an embodiment of the present invention.
  • the above-mentioned cancellation unit 104 includes :
  • the first cancellation subunit 1042 is configured to perform echo cancellation on the first echo signal according to the first reference signal
  • the second cancellation subunit 1044 is configured to perform echo cancellation on the second echo signal according to the second reference signal; wherein the second reference signal is generated according to the second audio signal.
  • first cancellation subunit and second cancellation subunit can respectively process the first echo signal and the second echo signal in the first audio signal.
  • the first cancellation subunit and the second cancellation subunit can adopt the layout shown in FIG. 8, so that the first audio signal passes through the first cancellation subunit and the second cancellation subunit in turn to perform the first echo signal and the second echo signal.
  • the first audio signal can also pass through the second cancellation subunit and the first cancellation subunit in sequence to cancel the second echo signal and the first echo signal, which is not limited in the present invention.
  • the cancellation of the first echo signal by the above-mentioned first cancellation subunit can be implemented by a DSP chip.
  • the first cancellation subunit 1042 performs inversion processing on the first reference signal to generate the first filtered signal, and combines the first audio The signal is superimposed with the first filtered signal to perform echo cancellation on the first echo signal.
  • FIG. 9 is a schematic diagram of the operation of a second reference signal according to an embodiment of the present invention.
  • the second reference signal can be obtained by the DSP chip before the second audio signal is sent to the screen sound unit for output. Signal and generate a second reference signal to send to the second cancellation subunit for echo cancellation.
  • the second elimination subunit 1044 is further configured to:
  • the first audio signal and the second filtered signal are superimposed to perform echo cancellation on the second echo signal.
  • the adaptive filter can be an adaptive filter configured in the terminal’s own processor, or a separately configured adaptive filter.
  • the adaptive filter is integrated in the audio processing In the DSP module.
  • the adaptive filter may perform filtering processing on the second reference signal through a preset algorithm to generate a second filtered signal.
  • the preset algorithm may be an adaptive algorithm or an acoustic echo cancellation (AEC) algorithm , Noise suppression (NS) algorithm, etc.
  • FIG. 10 is a working schematic diagram of the echo cancellation device according to an embodiment of the present invention.
  • the structure and work flow of the echo cancellation device in the embodiment of the present invention are shown in FIG. 10.
  • the output of the second audio signal in the terminal and the cancellation processing of the first echo signal and the second echo signal are all implemented by the DSP chip used for audio processing of the terminal.
  • the second audio signal pre-output by the terminal is sent to the exciter by the DSP chip to make the exciter vibrate and then drive the screen to vibrate to output the sound corresponding to the second audio signal; at the same time, in the above output process, the DSP chip can use the second audio
  • the signal generates a second reference signal.
  • the vibration sensor can detect the vibration parameters of the exciter in real time to obtain the vibration signal according to the vibration parameters, and then pass the vibration signal through the low noise amplifier (LNA), ADC, filter Fiter and other devices in turn After processing, the first reference signal is generated.
  • LNA low noise amplifier
  • ADC ADC
  • filter Fiter filter Fiter
  • vibration noise is generated when the exciter drives the terminal to vibrate, and this vibration noise is the first echo signal; on the other hand, the sound generated by the screen sound is transmitted through the air to form a second echo signal;
  • the first echo signal and the second echo signal are picked up by the microphone at the same time, and the target audio signal actively input by the user through the microphone is also picked up.
  • the target audio signal, the first echo signal, and the second echo signal together constitute the first audio signal Go to the pickup unit.
  • Fig. 11 is a schematic diagram of echo cancellation performed by the cancellation unit according to an embodiment of the present invention. As shown in Fig. 11, in the first cancellation unit, the above-mentioned first reference signal can be inverted through the DSP chip to obtain the first reference signal.
  • the waveform relationship between the first filtered signal and the first reference signal is shown in Figure 8; after the first audio signal enters the first cancellation subunit, the first filtered signal can be It is superimposed with the first audio signal to eliminate the first echo signal in the first audio signal.
  • the second reference signal can be processed by the adaptive filter set in the DSP chip according to the preset adaptive algorithm, AEC algorithm, and NS algorithm to obtain a phase opposite to the second reference signal. After the first audio signal enters the second cancellation subunit, the second filter signal can be superimposed with the first audio signal to eliminate the second echo signal in the first audio signal.
  • the target audio signal can be approximated; in this way, the target audio signal can be sent or stored, In order to avoid the inclusion of echoes.
  • An embodiment of the present invention also provides a sound pickup device, which is set in a terminal.
  • the terminal includes at least a screen sounding unit 106 and a first sound pickup unit 1082, and a second sound pickup unit 1084.
  • FIG. 12 is a terminal provided according to an embodiment of the invention. Schematic diagram of sound on the screen (3).
  • FIG. 13 is a diagram of sound on the screen of a terminal according to an embodiment of the present invention (4).
  • the first sound pickup unit 1082 in the terminal in the embodiment of the present invention The second pickup unit 1084 and the second pickup unit 1084 are respectively arranged at the top and bottom of the terminal.
  • Fig. 14 is a functional schematic diagram of a sound pickup device according to an embodiment of the present invention. As shown in Fig. 14, the sound pickup device in the embodiment of the present invention includes:
  • the detection unit 202 includes:
  • the first detection sub-unit 2022 is arranged within a preset range of the first pickup unit 1082, and the first detection sub-unit 2022 is configured to detect the vibration parameter of the area where the first pickup unit in the terminal is located to obtain the first vibration signal;
  • the second detection subunit 2024 is set within the preset range of the second pickup unit 1084, and the second detection subunit 2024 is configured to detect the vibration parameter of the area where the second pickup unit in the terminal is located to obtain the second vibration signal;
  • the echo cancellation device further includes:
  • the processing unit 204 is configured to select the first sound pickup unit or the second sound pickup unit to input the target audio according to the first vibration signal and the second vibration signal.
  • the sound pickup device in the embodiment of the present invention can be installed independently or integrated with the echo cancellation device, that is, the sound pickup device in this embodiment can be implemented by the echo cancellation device in cooperation.
  • the first detection unit/second detection unit included in the detection unit in the sound pickup device is constituted by the detection unit in the echo cancellation device, that is, the first detection unit included in the detection unit in the sound pickup device.
  • the unit/second detection unit can be respectively realized by a detection unit in an echo cancellation device, and the detection of vibration parameters is realized by the vibration sensor included in the detection unit in the foregoing embodiment of the present invention. Therefore, the sound pickup device in the embodiment of the present invention and the echo cancellation device can use the same detection unit and be installed in the terminal at the same time.
  • the remaining optional implementations and technical effects of the first detection subunit/second detection subunit included in the detection unit in the sound pickup device are all corresponding to the detection unit in the echo cancellation device, so I won't repeat it here.
  • the processing unit in the embodiment of the present invention may be implemented by a processor of the terminal itself.
  • the first detection subunit is arranged within the preset range of the first pickup unit, that is, it indicates that the first detection subunit is arranged closer to the first pickup unit than the second pickup unit.
  • the first The second detection subunit is arranged within the preset range of the second pickup unit, which means that the second detection subunit is arranged closer to the second pickup unit than the first pickup unit.
  • first detection subunit and the second detection subunit can be performed by the same detection device, such as a vibration sensor, for different areas of the terminal, that is, the area where the first pickup unit is located or the area where the second pickup unit is located.
  • one detection device can also be set in the above-mentioned different areas.
  • the first detection subunit and the second detection subunit in the detection unit can be used.
  • the unit respectively detects the terminal vibration parameters of the area where the first pickup unit and the second pickup unit are located to generate the corresponding first vibration signal and second vibration signal, and then compare the vibration conditions of the terminals in the area where the two pickup units are located .
  • one of the sound pickup units can be selected as the main sound pickup unit, that is, the sound pickup unit used to input the target audio, according to the vibration condition of the terminal in the area where the above two sound pickup units are located.
  • the above judgment process is as follows:
  • the second sound pickup unit 1084 is input to the target audio.
  • the sound pickup unit provided in an area with less vibration in the terminal can be used as the main sound pickup unit to input the target audio.
  • the selected main pickup unit has less vibration in its area, so the vibration noise generated by the terminal's vibration in this area is also smaller than that of the other pickup unit, that is, the first echo signal generated is higher than that of another pickup unit.
  • a sound pickup unit is small, so that the main sound pickup unit can further control the influence of the first echo signal contained in it on the target audio in the process of picking up the first audio signal.
  • the first sound pickup unit and the second sound pickup unit in the embodiment of the present invention can indicate any two sound pickup units among the plurality of sound pickup units, that is, for those with more than two sound pickup units.
  • the above solution can be used to determine a main pickup unit between any two pickup units in the multiple pickup units, and then the multiple pickup units can be compared in pairs to finally determine the multiple pickup units.
  • the terminal further includes a first screen sounding unit 1062 and a second sound pickup unit.
  • the two-screen sounding unit 1064 wherein the first-screen sounding unit 1062 is arranged within a preset range of the first sound pickup unit 1082, and the second-screen sounding unit 1064 is arranged within a preset range of the second sound pickup unit 1084.
  • the first detection subunit 2022 is set within the preset range of the first pickup unit 1062, and the first detection subunit is configured to detect the vibration parameter of the first screen sounding unit to obtain the first vibration signal;
  • the second detection subunit 2024 is disposed within the preset range of the second pickup unit 1064, and the second detection subunit is configured to detect the vibration parameter of the second screen sound unit to obtain the second vibration signal.
  • the embodiment of the present invention may further provide multiple screen sounding units, for example, multiple exciters, so as to realize partitioned sound or privacy protection during the screen sounding process.
  • the first detection subunit and the second detection subunit can directly detect the first pickup unit and the second pickup unit corresponding to the second pickup unit.
  • the vibration parameters of the one-screen sounding unit and the second-screen sounding unit are used to determine the vibration between the corresponding pickup units.
  • the first screen sounding unit is arranged within the preset range of the first sound pickup unit, that is, it indicates that the first screen sounding unit is arranged closer to the first sound pickup unit than the second sound pickup unit.
  • the two-screen sound generating unit is arranged within the preset range of the second sound pickup unit, which indicates that the second screen sound unit is arranged closer to the second sound pickup unit than the first sound pickup unit.
  • An embodiment of the present invention also provides a terminal, which includes the above-mentioned echo cancellation device, and/or, a sound pickup device. It should be noted that the remaining optional embodiments and technical implementations of the echo cancellation device and the sound pickup device in the terminal in the embodiment of the present invention all correspond to the foregoing echo cancellation device and the sound pickup device, so they will not be repeated here.
  • the embodiment of the present invention also provides an echo cancellation method, which is applied to a terminal, and the terminal includes at least a screen sounding unit and a pickup unit.
  • FIG. 16 is a flowchart of the echo cancellation method according to the embodiment of the present invention, as shown in FIG. 16.
  • the echo cancellation method in the embodiment of the present invention includes:
  • S102 Detect vibration parameters of the screen sounding unit to obtain a vibration signal, and generate a first reference signal according to the vibration signal;
  • S104 Perform echo cancellation on the first echo signal in the first audio signal according to the first reference signal; wherein the first audio signal is an audio signal picked up by the pickup unit, and the first audio signal includes at least the first echo signal, and An echo signal is formed by the noise generated by the screen sound unit driving the terminal to vibrate.
  • performing echo cancellation on the first echo signal in the first audio signal according to the first reference signal includes:
  • the first audio signal and the first filtered signal are superimposed to perform echo cancellation on the first echo signal.
  • the first audio signal includes: a first echo signal and a second echo signal
  • the second echo signal is formed by the sound output by the screen sound unit driving the terminal to vibrate according to the second audio signal, and the second audio signal is an audio signal pre-output by the terminal.
  • performing echo cancellation on the first echo signal in the first audio signal according to the first reference signal includes:
  • the foregoing performing echo cancellation on the first echo signal according to the first reference signal includes:
  • the first audio signal and the first filtered signal are superimposed to perform echo cancellation on the first echo signal.
  • the foregoing performing echo cancellation on the second echo signal according to the second reference signal includes:
  • the first audio signal and the second filtered signal are superimposed to perform echo cancellation on the second echo signal.
  • the terminal includes a first sound pickup unit and a second sound pickup unit; the echo cancellation method in the embodiment of the present invention further includes:
  • the first pickup unit or the second pickup unit is selected to input the target audio.
  • the terminal further includes a first screen sounding unit and a second screen sounding unit, wherein the first screen sounding unit is arranged within a preset range of the first sound pickup unit, and the second screen sounding unit is arranged for Within the preset range of the second pickup unit;
  • the above detecting the vibration parameter of the area where the first sound pickup unit in the terminal is located to obtain the first vibration signal, and detecting the vibration parameter of the area where the second sound pickup unit in the terminal is located to obtain the second vibration signal includes:
  • the vibration parameter of the first screen sound unit is detected to obtain the first vibration signal
  • the vibration parameter of the second screen sound unit is detected to obtain the second vibration signal
  • the foregoing selecting the first sound pickup unit or the second sound pickup unit to input the target audio according to the first vibration signal and the second vibration signal includes:
  • the second sound pickup unit is input to the target audio.
  • the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is Better implementation.
  • the technical solution of the present invention essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in each embodiment of the present invention.
  • An embodiment of the present invention also provides a sound pickup method, which is applied to a terminal.
  • the terminal includes at least a screen sounding unit, a first sound pickup unit, and a second sound pickup unit.
  • the flow chart, as shown in FIG. 17, the sound pickup method in the embodiment of the present invention includes:
  • S202 Detect the vibration parameter of the area where the first sound pickup unit in the terminal is located to obtain a first vibration signal, and detect the vibration parameter of the area where the second sound pickup unit in the terminal is located to obtain the second vibration signal;
  • S204 According to the first vibration signal and the second vibration signal, select the first sound pickup unit or the second sound pickup unit to input the target audio.
  • the terminal further includes a first screen sounding unit and a second screen sounding unit, wherein the first screen sounding unit is arranged within a preset range of the first sound pickup unit, and the second screen sounding unit is arranged for Within the preset range of the second pickup unit;
  • detecting the vibration parameter of the area where the first sound pickup unit in the terminal is located to obtain the first vibration signal, and detecting the vibration parameter of the area where the second sound pickup unit in the terminal is located to obtain the second vibration signal includes:
  • the vibration parameter of the first screen sound unit is detected to obtain the first vibration signal
  • the vibration parameter of the second screen sound unit is detected to obtain the second vibration signal
  • selecting the first sound pickup unit or the second sound pickup unit to input the target audio according to the first vibration signal and the second vibration signal includes:
  • the second sound pickup unit is input to the target audio.
  • the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is Better implementation.
  • the technical solution of the present invention essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in each embodiment of the present invention.
  • the embodiment of the present invention also provides a computer-readable storage medium in which a computer program is stored, wherein the computer program is configured to execute the steps in any one of the foregoing method embodiments when running.
  • the above-mentioned computer-readable storage medium may include, but is not limited to: U disk, Read-Only Memory (Read-Only Memory, ROM for short), Random Access Memory (Random Access Memory, RAM for short) , Mobile hard drives, magnetic disks or optical discs and other media that can store computer programs.
  • U disk Read-Only Memory
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • Mobile hard drives magnetic disks or optical discs and other media that can store computer programs.
  • An embodiment of the present invention also provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the foregoing method embodiments.
  • the aforementioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the aforementioned processor, and the input-output device is connected to the aforementioned processor.
  • modules or steps of the present invention can be implemented by a general computing device, and they can be concentrated on a single computing device or distributed in a network composed of multiple computing devices. Above, they can be implemented with program codes executable by a computing device, so that they can be stored in a storage device for execution by the computing device, and in some cases, they can be executed in a different order than shown here. Or the described steps, or fabricate them into individual integrated circuit modules respectively, or fabricate multiple modules or steps of them into a single integrated circuit module to achieve. In this way, the present invention is not limited to any specific combination of hardware and software.
  • the embodiments of the present invention applied to the audio processing field, can solve the problem that the noise generated by the terminal vibration during the screen sound process cannot be effectively eliminated in the related art, so as to eliminate the noise generated by the terminal vibration during the screen sound process and improve the terminal input audio frequency.
  • the effect of sound quality can solve the problem that the noise generated by the terminal vibration during the screen sound process cannot be effectively eliminated in the related art, so as to eliminate the noise generated by the terminal vibration during the screen sound process and improve the terminal input audio frequency.

Abstract

本发明实施例提供了一种回声消除装置及方法、拾音装置及方法、终端，其中，回声消除装置设置于终端，终端至少包括屏幕发声单元与拾音单元，装置包括：检测单元，配置为检测屏幕发声单元的振动参数以得到振动信号，并根据振动信号生成第一参考信号；消除单元，配置为根据第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，第一音频信号为拾音单元拾取的音频信号，第一音频信号中至少包括第一回声信号，第一回声信号由屏幕发声单元驱动终端振动所产生的噪音形成。通过本发明，解决了相关技术中屏幕发声过程中对于终端振动产生的噪声无法有效消除的问题，以达到消除屏幕发声过程中终端振动产生的噪声的效果。

Description

回声消除装置及方法、拾音装置及方法、终端 技术领域
本发明实施例涉及音频处理领域，具体而言，涉及一种回声消除装置及方法、拾音装置及方法、终端。
背景技术
随着终端对屏占比需求的发展，越来越多的终端采用屏幕发声技术实现发声。屏幕发声技术是通过一个屏幕发声器件驱动前方的屏幕及结构，以屏幕作为振动体，借由振动产生声波发出声音，其原理是将震动机械能直接传输到屏幕，然后让屏幕代替传统扬声器振膜发声。屏幕振动发声过程中，除屏幕振动产生相应的声音外，终端的整个结构都会在屏幕发声器件的作用下振动，该振动会产生一定的原始噪声，终端的拾音单元，如麦克风，则会在该过程中拾取到上述振动产生的原始噪声，由于该原始噪声缺少参考信号且属于非线性噪声，故无法采用相关技术中的降噪模块和回声模块直接进行消除，进而造成终端屏幕发声过程中存在回声。
针对上述相关技术中，屏幕发声过程中对于终端振动产生的噪声无法有效消除的问题，相关技术中尚未提出有效的解决方案。
发明内容
本发明实施例提供了一种回声消除装置及方法、拾音装置及方法、终端，以至少解决相关技术中屏幕发声过程中对于终端振动产生的噪声无法有效消除的问题。
根据本发明的一个实施例，提供了一种回声消除装置，设置于终端，所述终端至少包括屏幕发声单元与拾音单元，所述装置包括：
检测单元，配置为检测所述屏幕发声单元的振动参数以得到振动信号，并根据所述振动信号生成第一参考信号；
消除单元，配置为根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，所述第一音频信号为所述拾音单元拾取的音频信号，所述第一音频信号中至少包括第一回声信号，所述第一回声信号由所述屏幕发声单元驱动所述终端振动所产生的噪音形成。
根据本发明的另一个实施例，还提供了一种拾音装置，设置于终端，所述终端至少包括屏幕发声单元与第一拾音单元、第二拾音单元，所述装置包括：
检测单元，所述检测单元还包括：
第一检测子单元，设置于所述第一拾音单元的预设范围内，所述第一检测子单元配置为，检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号；
第二检测子单元，设置于所述第二拾音单元的预设范围内，所述第二检测子单元配置为，检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
所述装置还包括：
处理单元，配置为根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元 或所述第二拾音单元输入目标音频。
根据本发明的另一个实施例，还提供了一种终端，包括上述实施例中记载的回声消除装置，和/或，拾音装置。
根据本发明的另一个实施例，还提供了一种回声消除方法，应用于终端，所述终端至少包括屏幕发声单元与拾音单元，所述方法包括：
检测所述屏幕发声单元的振动参数以得到振动信号，并根据所述振动信号生成第一参考信号；
根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，所述第一音频信号为所述拾音单元拾取的音频信号，所述第一音频信号中至少包括第一回声信号，所述第一回声信号由所述屏幕发声单元驱动所述终端振动所产生的噪音形成。
根据本发明的另一个实施例，还提供了一种拾音方法，应用于终端，所述终端至少包括屏幕发声单元、第一拾音单元与第二拾音单元，所述方法还包括：
检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号，并检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频。
根据本发明的另一个实施例，还提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
根据本发明的另一个实施例，还提供了一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行上述任一项方法实施例中的步骤。
通过本发明实施例，由于可通过检测单元检测屏幕发声单元的振动参数以得到振动信号，并根据振动信号生成第一参考信号；进而可通过消除单元根据第一参考信号对拾音单元所拾取的第一音频信号中的第一回声信号进行回声消除；上述第一回声信号即由屏幕发声单元驱动终端振动所产生的噪音形成。因此，本发明实施例可以解决相关技术中屏幕发声过程中对于终端振动产生的噪声无法有效消除的问题，以达到消除屏幕发声过程中终端振动产生的噪声，提高终端输入音频的音质的效果。
附图说明
图1是根据本发明实施例提供的回声消除装置的应用场景示意图；
图2是根据本发明实施例提供的终端的硬件结构框图；
图3是根据本发明实施例提供的终端屏幕发声的示意图(一)；
图4是根据本发明实施例提供的终端屏幕发声的示意图(二)；
图5是根据本发明实施例提供的回声消除装置的功能示意图(一)；
图6是根据本发明实施例提供的回声消除装置的功能示意图(二)；
图7是根据本发明实施例提供的消除单元的工作示意图；
图8是根据本发明实施例提供的回声消除装置的功能示意图(三)；
图9是根据本发明实施例提供的第二参考信号的工作示意图；
图10是根据本发明实施例提供的回声消除装置的工作示意图；
图11是根据本发明实施例提供的消除单元进行回声消除的示意图；
图12是根据本发明实施例提供的终端屏幕发声的示意图(三)；
图13是根据本发明实施例提供的终端屏幕发声的示意图(四)；
图14是根据本发明实施例提供的拾音装置的功能示意图；
图15是根据本发明实施例提供的终端屏幕发声的示意图(五)；
图16是根据本发明实施例提供的回声消除方法的流程图；
图17是根据本发明实施例提供的拾音方法的流程图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本发明的实施例。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。
为进一步阐述本发明实施例中的回声消除装置及方法、拾音装置及方法、终端，以下对本发明实施例中的回声消除装置及方法、拾音装置及方法、终端的应用场景进行说明：
图1是根据本发明实施例提供的回声消除装置的应用场景示意图，如图1所示，用户A使用终端与用户B进行语音通话，该终端为通过屏幕发声单元，如悬臂压电陶瓷、激励器等产生的振动以驱使屏幕振动发声的终端。终端接收到用户B发送的语音消息1，通过屏幕发声单元振动以驱使屏幕振动输出语音消息1令用户A收听；同时，用户A通过终端的拾音单元，如麦克风等，输入语音消息2，终端将上述语音消息2发送至用户B。
终端在输出语音消息1的过程中，屏幕发声单元产生的振动一方面可驱使屏幕振动发声，另一方面会令终端整体产生振动，进而形成一定幅度的振动噪音。因此，用户A通过终端的拾音单元输入语音消息2的过程中，除用户A输入的语音消息2外，终端的拾音单元会同时拾取到上述振动噪音，进而发送至用户B。因此，上述终端的拾音单元所拾取并发送至用户B的语音消息，实际为携带了振动噪音的音频信息。对于用户B而言，由于其接收到的音频信息中夹带有振动噪音，故会较大的影响其收听效果。
本发明实施例中的终端可以为移动终端、平板电脑、计算机终端、可穿戴终端等任何可实现屏幕发声的设备。以运行在移动终端上为例，图2是根据本发明实施例提供的终端的硬件结构框图，如图2所示，移动终端可以包括一个或多个(图2中仅示出一个)处理器002(处理器002可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)和用于存储数据的存储器004，其中，上述移动终端还可以包括用于通信功能的传输设备006以及输入输出设备008。本领域普通技术人员可以理解，图2所示的结构仅为示意，其并不对上述移动终端的结构造成限定。例如，移动终端还可包括比图2中所示更多或者更少的组件，或者具有与图1所示不同的配置。
存储器004可用于存储计算机程序，例如，应用软件的软件程序以及模块，如本发明实施例中的回声消除方法与拾音方法对应的计算机程序，处理器002通过运行存储在存储器004内的计算机程序，从而执行各种功能应用以及数据处理，即实现上述的方法。存储器004可包括高速随机存储器，还可包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器004可进一步包括相对于处理器002远程设置的存储器，这些远程存储器可以通过网络连接至移动终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
传输装置006用于经由一个网络接收或者发送数据。上述的网络具体实例可包括移动终端的通信供应商提供的无线网络。在一个实例中，传输装置006包括一个网络适配器(Network Interface Controller，简称为NIC)，其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中，传输装置006可以为射频(Radio Frequency，简称为RF)模块，其用于通过无线方式与互联网进行通讯。
对此，本发明实施例提供了一种回声消除装置及方法、拾音装置及方法、终端，以有效消除屏幕发声过程中终端振动产生的噪声，提高终端输入音频的音质的效果。以下对于本发明实施例中的回声消除装置及方法、拾音装置及方法、终端进行阐述：
本发明实施例提供了一种回声消除装置，设置于终端，终端至少包括屏幕发声单元106与拾音单元108。图3是根据本发明实施例提供的终端屏幕发声的示意图(一)，图4是根据本发明实施例提供的终端屏幕发声的示意图(二)，如图3与图4所示，本发明实施例中的终端通过屏幕发声单元振动，驱使屏幕振动发声。图5是根据本发明实施例提供的回声消除装置的功能示意图(一)，如图5所示，所述装置包括：
检测单元102，配置为检测屏幕发声单元的振动参数以得到振动信号，并根据振动信号生成第一参考信号；
消除单元104，配置为根据第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，第一音频信号为拾音单元拾取的音频信号，第一音频信号中至少包括第一回声信号，第一回声信号由屏幕发声单元驱动终端振动所产生的噪音形成。
需要说明的是，在一示例中，上述屏幕发声单元可以为悬臂压电陶瓷或激励器，上述拾音单元可以为麦克风。本发明实施例中，屏幕发声单元的振动参数用于指示屏幕发声单元为输出音频信息而产生振动以驱使屏幕振动发声过程中的振动参数，该振动参数可以为屏幕发声单元的振动频率、振动幅度等，本发明对此不做限定。
本发明实施例中，检测单元设置于终端内部，图6是根据本发明实施例提供的回声消除装置的功能示意图(二)，如图6所示，在一可选实施例中，上述检测单元102包括：
振动传感子单元1022，设置于屏幕发声单元的预设范围内，振动传感子单元配置为，检测屏幕发声单元的振动参数以得到振动信号；
模数转换(Analogue-to-Digital Conversion，ADC)子单元1024，配置为对振动信号进行模数转换处理，以得到第一参考信号。
需要说明的是，上述振动传感子单元可以由振动传感器构成，在一示例中，振动传感器可以为电感式振动传感器，其可通过电磁感应，利用线圈自感或互感系数的变化来实现对于屏幕发声单元的振动参数的检测，进而将屏幕发声单元振动过程中的机械振动参数转化为电参量信号，即上述振动信号。由于振动传感子单元获取的振动信号为模拟信号，故需通过ADC子单元对该振动进行进行模数转换处理，以得到第一参考信号。
需要说明的是，为有效检测屏幕发声单元的振动参数，振动传感子单元需设置于屏幕发声单元的预设范围内，本领域技术人员可以根据终端内部布局与振动传感子单元的检测方式确定上述预设范围，在一示例中，振动传感子单元设置在屏幕发声单元的侧端，并紧邻屏幕发声单元设置。
本发明实施例中，上述屏幕发声单元振动过程中，除却驱使屏幕振动发声外，屏幕发声单元的振动亦会导致终端整体振动，进而令终端振动过程中产生振动噪音，该振动噪音即为 上述第一回声信息。本发明实施例中，上述第一回声信息可被拾音单元拾取，进而可通过上述消除单元对其进行消除。在一可选实施例中，上述消除单元104配置为，
对第一参考信号进行反相处理以生成第一滤波信号；
将第一音频信号与第一滤波信号叠加，以对第一回声信号进行回声消除。
需要说明的是，上述对于第一参考信号进行反相处理，即获取与第一参考信号相位相反的信号，该信号即为第一滤波信号；通过将该第一滤波信号与夹带有第一回声信号的第一音频信号叠加，即可通过第一滤波信号与第一回声信号的抵消，以消除第一音频信号中的第一回声信号。本发明实施例中，消除单元可以由用于音频处理的数字信号处理(Digital Signal Process，DSP)芯片构成，通过DSP芯片对于第一音频信号进行DSP处理，即上述对第一参考信号进行反相处理以生成第一滤波信号；以及将第一音频信号与第一滤波信号叠加，以对第一回声信号进行回声消除。
需要说明的是，本发明实施例中的消除单元可以由独立的DSP芯片构成，也可以集成在终端自身用于音频处理的DSP芯片或其它处理器中，本发明对此不做限定。
图7是根据本发明实施例提供的消除单元的工作示意图，如图7所示，检测单元中的振动传感器检测至屏幕发声单元的振动信号，并通过ADC进行模数转换处理后，以得到第一参考信号，消除单元即可对上述第一参考信号进行反相处理以生成第一滤波信号；第一参考信号与第一滤波信号的波形即如图7所示，第一参考信号与第一滤波信号的相位相反。
在一可选实施例中，上述第一音频信号包括：第一回声信号、第二回声信号；
其中，第二回声信号由屏幕发声单元根据第二音频信号驱动终端振动所输出的声音形成，第二音频信号为终端预输出的音频信号。
需要说明的是，在一示例中，第二音频信息可以为其它用户发送至终端的语音信号，或者，终端预备播放的服务提供商提供的音频/视频信息，即终端预备进行输出/播放的音频即为第二音频信息。当终端预输出第二音频信号的情形下，终端可根据第二音频信号控制屏幕发声单元进行振动，以驱使屏幕随之振动以输出期望的声音。
上述终端输出声音的过程中，屏幕输出的声音一方面可被用户收听，同时亦会被终端的拾音单元拾取，尤其在室内或狭窄环境内，上述屏幕输出的声音被终端的拾音单元拾取的过程可类似于回声传递的过程，拾音单元所拾取的由于屏幕发声单元驱动终端振动所输出的声音的回传而形成的声音信号，即为本发明实施例中的第二回声信号。
需要说明的是，上述可选实施例中，第一音频信号中除第一回声信号与第二回声信号，还包括目标音频信号，目标音频信号为通过拾音单元输入的目标音频，即用户通过拾音单元主动输入的目标音频，在一示例中，用户与其它用户进行语音通话过程中，目标音频即为用户通过终端发送至第三方的语音消息；在另一示例中，用户通过终端进行录音过程中，目标音频即为用户通过终端录取的现场声音。上述终端的拾音单元在拾取目标音频信号的过程中，实际拾取的音频信号中携带了上述第一回声信号与上述第二回声信号，即上述可选实施例中，包括第一回声信号、第二回声信号、目标音频信号的第一音频信号。
为消除上述第一音频信号中的第一回声信号与第二回声信号，图8是根据本发明实施例提供的回声消除装置的功能示意图(三)，如图8所示，上述消除单元104包括：
第一消除子单元1042，配置为根据第一参考信号对第一回声信号进行回声消除；
第二消除子单元1044，配置为根据第二参考信号对第二回声信号进行回声消除；其中， 第二参考信号是根据第二音频信号生成的。
需要说明的是，上述第一消除子单元与第二消除子单元可分别对于第一音频信号中的第一回声信号与第二回声信号进行处理。第一消除子单元与第二消除子单元可以采用如图8所示的布局方式，令第一音频信号依次通过第一消除子单元与第二消除子单元进行第一回声信号与第二回声信号消除，也可以令第一音频信号依次通过第二消除子单元与第一消除子单元进行第二回声信号与第一回声信号的消除，本发明对此不做限定。
上述第一消除子单元对第一回声信号进行的消除，可由DSP芯片实现，具体即为第一消除子单元1042对第一参考信号进行反相处理以生成第一滤波信号，并将第一音频信号与第一滤波信号叠加，以对第一回声信号进行回声消除。
上述第二消除子单元对第二回声信号进行的消除，是基于第二参考信号实现的。图9是根据本发明实施例提供的第二参考信号的工作示意图，如图9所示，第二参考信号可由DSP芯片在第二音频信号发送至屏幕发声单元进行输出前，获取该第二音频信号，并以此生成第二参考信号，以发送至第二消除子单元供回声消除使用。在一可选实施例中，第二消除子单元1044还配置为，
通过自适应滤波器对第二参考信号进行滤波处理，以生成第二滤波信号；
将第一音频信号与第二滤波信号叠加，以对第二回声信号进行回声消除。
需要说明的是，自适应滤波器可以为终端自身的处理器中配置的自适应滤波器，也可以是单独配置的自适应滤波器，通常而言，自适应滤波器集成于用于音频处理的DSP模块中。自适应滤波器可通过预设的算法对第二参考信号进行滤波处理以生成第二滤波信号，在一示例中，预设算法可以为自适应算法、声学回声消除(Acoustic Echo Cancellation，AEC)算法、噪声抑制(noise suppression，NS)算法等。
图10是根据本发明实施例提供的回声消除装置的工作示意图，本发明实施例中的回声消除装置的构架与工作流程如图10所示。如图10所示，本发明实施例中，终端中第二音频信号的输出，以及对于第一回声信号与第二回声信号的消除处理均由终端用于音频处理的DSP芯片实现。终端预输出的第二音频信号由DSP芯片发送至激励器，以令激励器振动进而驱使屏幕振动发声以输出第二音频信号对应的声音；同时，在上述输出过程中，DSP芯片可由第二音频信号生成第二参考信号。
激励器工作过程中，振动传感器可实时检测激励器的振动参数，以根据该振动参数获取振动信号，将该振动信号依次经由低噪声放大器(low noise amplifier，LNA)，ADC，滤波器Fiter等器件处理后以生成第一参考信号。
屏幕振动发声过程中，一方面激励器驱使终端振动过程中会产生振动噪音，该振动噪音即为第一回声信号；另一方面，屏幕发声产生的声音经由空气传播，形成第二回声信号；上述第一回声信号与第二回声信号同时被麦克风拾取，同时被拾取的还有用户通过麦克风主动输入的目标音频信号，上述目标音频信号、第一回声信号、第二回声信号共同构成第一音频信号进入至拾取单元。
拾取单元拾取第一音频信号后，将第一音频信号依次经由LNA、ADC、Fiter等器件处理后送入至消除单元。消除单元中的第一消除子单元EC1根据第一参考信号对于第一回声信号进行消除，第二消除子单元EC2根据第二参考信号对于第二回声信号进行消除。图11是根据本发明实施例提供的消除单元进行回声消除的示意图，如图11所示，第一消除单元中，可 通过DSP芯片对上述第一参考信号进行反相处理，以得到与第一参考信号相位相反的第一滤波信号，第一滤波信号与第一参考信号的波形关系即如图8所示；第一音频信号进入至第一消除子单元后，即可将上述第一滤波信号与第一音频信号叠加，以消除第一音频信号中的第一回声信号。第二消除子单元中，可通过设置于DSP芯片中的自适应滤波器根据预设的自适应算法、AEC算法、NS算法对于上述第二参考信号进行处理，以得到与第二参考信号相位相反的第二滤波信号，第一音频信号进入至第二消除子单元后，即可将上述第二滤波信号与第一音频信号叠加，以消除第一音频信号中的第二回声信号。
第一音频信号消除上述第一回声信号与第二回声信号后，进一步进行降噪与补充增益等处理后，即可近似得到目标音频信号；以此，即可对目标音频信号进行发送或存储，以避免其中夹杂回声。
本发明实施例还提供了一种拾音装置，设置于终端，终端至少包括屏幕发声单元106与第一拾音单元1082、第二拾音单元1084，图12是根据本发明实施例提供的终端屏幕发声的示意图(三)，图13是根据本发明实施例提供的终端屏幕发声的示意图(四)，如图12与图13所示，本发明实施例中的终端中第一拾音单元1082与第二拾音单元1084分别设置于终端的顶端与底端。图14是根据本发明实施例提供的拾音装置的功能示意图，如图14所示，本发明实施例中的拾音装置包括：
检测单元202，检测单元202包括：
第一检测子单元2022，设置于第一拾音单元1082的预设范围内，第一检测子单元2022配置为，检测终端中第一拾音单元所在区域的振动参数以得到第一振动信号；
第二检测子单元2024，设置于第二拾音单元1084的预设范围内，第二检测子单元2024配置为，检测终端中第二拾音单元所在区域的振动参数以得到第二振动信号；
本发明实施例中，回声消除装置还包括：
处理单元204，配置为根据第一振动信号与第二振动信号，选取第一拾音单元或第二拾音单元输入目标音频。
需要说明的是，本发明实施例中的拾音装置可以独立设置，也可以与回声消除装置集成设置，即本实施例中的拾音装置可以由回声消除装置配合实现。在一示例中，上述拾音装置中的检测单元所包含的第一检测单元/第二检测单元由回声消除装置中的检测单元构成，即上述拾音装置中的检测单元所包含的第一检测单元/第二检测单元，可分别由一个回声消除装置中的检测单元实现，通过前述本发明实施例中检测单元所包含的振动传感器实现振动参数的检测。因此，本发明实施例中的拾音装置可以与回声消除装置采用同一检测单元，并同时设置在终端中。本发明实施例中，拾音装置中的检测单元所包含的第一检测子单元/第二检测子单元的其余可选实施方式以及技术效果，均与回声消除装置中的检测单元相对应，故在此不再赘述。此外，本发明实施例中的处理单元可以由终端自身的处理器实现。
需要说明的是，第一检测子单元设置于第一拾音单元的预设范围内，即指示第一检测子单元相较于第二拾音单元靠近第一拾音单元设置，类似的，第二检测子单元设置于第二拾音单元的预设范围内，即指示第二检测子单元相较于第一拾音单元靠近第二拾音单元设置。
需要说明的是，第一检测子单元与第二检测子单元可以由同一个检测器件，如振动传感器对于终端的不同区域，即上述第一拾音单元所在区域或第二拾音单元所在区域进行检测， 也可以在上述不同区域分别设置一个检测器件。
需要说明的是，对于同时设置第一拾音单元与第二拾音单元的终端，如采用双麦克风的终端，本发明实施例中可通过检测单元中的第一检测子单元与第二检测子单元分别检测第一拾音单元与第二拾音单元所在区域的终端振动参数，以生成对应的第一振动信号与第二振动信号，进而以此比较两个拾音单元所在区域终端的振动情况。通过上述方案，可进一步根据上述两个拾音单元所在区域终端的振动情况以选取其中的一个拾音单元作为主拾音单元，即用于输入目标音频的拾音单元。在一可选实施例中，上述判断的过程如下所示：
在第一振动信号所指示的振动频率小于第二振动信号所指示的振动频率的情形下，将第一拾音单元1082输入目标音频；或者，
在第一振动信号所指示的振动频率大于第二振动信号所指示的振动频率的情形下，将第二拾音单元1084输入目标音频。
需要说明的是，通过上述方案，即可采用设置于终端中振动较小的区域的拾音单元作为主拾音单元进行目标音频的输入。以此，选取的主拾音单元由于其所在区域振动较小，故终端在该区域振动产生的振动噪音亦较于另一拾音单元所在区域较小，即产生的第一回声信号较于另一拾音单元较小，进而使得主拾音单元在拾取第一音频信号过程中，其所包含的第一回声信号对目标音频的影响可得以进一步的控制。
需要说明的是，本发明实施例中的第一拾音单元与第二拾音单元可以指示多个拾音单元中的任意两个拾音单元，即对于设置有多于两个拾音单元的终端而言，可通过上述方案以在多个拾音单元中的任意两个拾音单元之间确定一主拾音单元，进而在多个拾音单元中通过两两比较的方式，最终确定多个拾音单元中的主拾音单元。
图15是根据本发明实施例提供的终端屏幕发声的示意图(五)，如图15所示，针对上述第一拾音单元与第二拾音单元，终端还包括第一屏幕发声单元1062与第二屏幕发声单元1064，其中，第一屏幕发声单元1062设置于第一拾音单元1082的预设范围内，第二屏幕发声单元1064设置于第二拾音单元1084的预设范围内。上述检测单元202中，
第一检测子单元2022，设置于第一拾音单元1062的预设范围内，第一检测子单元配置为，检测第一屏幕发声单元的振动参数以得到第一振动信号；
第二检测子单元2024，设置于第二拾音单元1064的预设范围内，第二检测子单元配置为，检测第二屏幕发声单元的振动参数以得到第二振动信号。
需要说明的是，本发明实施例可进一步设置多个屏幕发声单元，例如，多个激励器，以实现屏幕发声过程中的分区发声或隐私保护。以此，在设置第一拾音单元与第二拾音单元的过程中，即可由第一检测子单元与第二检测子单元直接检测第一拾音单元与第二拾音单元所对应的第一屏幕发声单元与第二屏幕发声单元的振动参数，以确定相应的拾音单元之间的振动情况。需要说明的是，第一屏幕发声单元设置于第一拾音单元的预设范围内，即指示第一屏幕发声单元相较于第二拾音单元靠近第一拾音单元设置，类似的，第二屏幕发声单元设置于第二拾音单元的预设范围内，即指示第二屏幕发声单元相较于第一拾音单元靠近第二拾音单元设置。
本发明实施例还提供了一种终端，该终端包括了上述回声消除装置，和/或，拾音装置。需要说明的是，本发明实施例中的终端中的回声消除装置与拾音装置的其余可选实施例与技 术实现均与前述回声消除装置与拾音装置对应，故在此不再赘述。
本发明实施例还提供了一种回声消除方法，应用于终端，终端至少包括屏幕发声单元与拾音单元，图16是根据本发明实施例提供的回声消除方法的流程图，如图16所示，本发明实施例中的回声消除方法包括：
S102，检测屏幕发声单元的振动参数以得到振动信号，并根据振动信号生成第一参考信号；
S104，根据第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，第一音频信号为拾音单元拾取的音频信号，第一音频信号中至少包括第一回声信号，第一回声信号由屏幕发声单元驱动终端振动所产生的噪音形成。
需要说明的是，本发明实施例中的回声消除方法的其余可选实施例与技术实现，均与上述回声消除装置对应，故在此不再赘述。
在一可选实施例中，上述S104中，根据第一参考信号对第一音频信号中的第一回声信号进行回声消除，包括：
对第一参考信号进行反相处理以生成第一滤波信号；
将第一音频信号与第一滤波信号叠加，以对第一回声信号进行回声消除。
在一可选实施例中，第一音频信号包括：第一回声信号、第二回声信号；
其中，第二回声信号由屏幕发声单元根据第二音频信号驱动终端振动所输出的声音形成，第二音频信号为终端预输出的音频信号。
在一可选实施例中，上述S104中，根据第一参考信号对第一音频信号中的第一回声信号进行回声消除，包括：
根据第一参考信号对第一回声信号进行回声消除，并根据第二参考信号对第二回声信号进行回声消除；其中，第二参考信号是根据第二音频信号生成的。
在一可选实施例中，上述根据第一参考信号对第一回声信号进行回声消除，包括：
对第一参考信号进行反相处理以生成第一滤波信号；
将第一音频信号与第一滤波信号叠加，以对第一回声信号进行回声消除。
在一可选实施例中，上述根据第二参考信号对第二回声信号进行回声消除，包括：
通过自适应滤波器对第二参考信号进行滤波处理，以生成第二滤波信号；
将第一音频信号与第二滤波信号叠加，以对第二回声信号进行回声消除。
在一可选实施例中，终端包括第一拾音单元与第二拾音单元；本发明实施例中的回声消除方法还包括：
检测终端中第一拾音单元所在区域的振动参数以得到第一振动信号，并检测终端中第二拾音单元所在区域的振动参数以得到第二振动信号；
根据第一振动信号与第二振动信号，选取第一拾音单元或第二拾音单元输入目标音频。
在一可选实施例中，终端还包括第一屏幕发声单元与第二屏幕发声单元，其中，第一屏幕发声单元设置于第一拾音单元的预设范围内，第二屏幕发声单元设置于第二拾音单元的预设范围内；
上述检测终端中第一拾音单元所在区域的振动参数以得到第一振动信号，并检测终端中第二拾音单元所在区域的振动参数以得到第二振动信号，包括：
检测第一屏幕发声单元的振动参数以得到第一振动信号，并检测第二屏幕发声单元的振动参数以得到第二振动信号。
在一可选实施例中，上述根据第一振动信号与第二振动信号，选取第一拾音单元或第二拾音单元输入目标音频，包括：
在第一振动信号所指示的振动频率小于第二振动信号所指示的振动频率的情形下，将第一拾音单元输入目标音频；或者，
在第一振动信号所指示的振动频率大于第二振动信号所指示的振动频率的情形下，将第二拾音单元输入目标音频。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
本发明实施例还提供了一种拾音方法，应用于终端，终端至少包括屏幕发声单元、第一拾音单元与第二拾音单元，图17是根据本发明实施例提供的拾音方法的流程图，如图17所示，本发明实施例中的拾音方法包括：
S202，检测终端中第一拾音单元所在区域的振动参数以得到第一振动信号，并检测终端中第二拾音单元所在区域的振动参数以得到第二振动信号；
S204，根据第一振动信号与第二振动信号，选取第一拾音单元或第二拾音单元输入目标音频。
需要说明的是，本发明实施例中的拾音方法的其余可选实施例与技术实现，均与上述拾音装置对应，故在此不再赘述。
在一可选实施例中，终端还包括第一屏幕发声单元与第二屏幕发声单元，其中，第一屏幕发声单元设置于第一拾音单元的预设范围内，第二屏幕发声单元设置于第二拾音单元的预设范围内；
上述S202中，检测终端中第一拾音单元所在区域的振动参数以得到第一振动信号，并检测终端中第二拾音单元所在区域的振动参数以得到第二振动信号，包括：
检测第一屏幕发声单元的振动参数以得到第一振动信号，并检测第二屏幕发声单元的振动参数以得到第二振动信号。
在一可选实施例中，上述S204中，根据第一振动信号与第二振动信号，选取第一拾音单元或第二拾音单元输入目标音频，包括：
在第一振动信号所指示的振动频率小于第二振动信号所指示的振动频率的情形下，将第一拾音单元输入目标音频；或者，
在第一振动信号所指示的振动频率大于第二振动信号所指示的振动频率的情形下，将第二拾音单元输入目标音频。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前 者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
本发明实施例还提供了一种计算机可读存储介质，该计算机可读存储介质中存储有计算机程序，其中，该计算机程序被设置为运行时执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述计算机可读存储介质可以包括但不限于：U盘、只读存储器(Read-Only Memory，简称为ROM)、随机存取存储器(Random Access Memory，简称为RAM)、移动硬盘、磁碟或者光盘等各种可以存储计算机程序的介质。
本发明实施例还提供了一种电子装置，包括存储器和处理器，该存储器中存储有计算机程序，该处理器被设置为运行计算机程序以执行上述任一项方法实施例中的步骤。
在一个示例性实施例中，上述电子装置还可以包括传输设备以及输入输出设备，其中，该传输设备和上述处理器连接，该输入输出设备和上述处理器连接。
本实施例中的具体示例可以参考上述实施例及示例性实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本发明的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。
工业实用性
本发明实施例，应用于音频处理领域，可以解决相关技术中屏幕发声过程中对于终端振动产生的噪声无法有效消除的问题，以达到消除屏幕发声过程中终端振动产生的噪声，提高终端输入音频的音质的效果。

Claims (28)

  1. 一种回声消除装置，设置于终端，所述终端至少包括屏幕发声单元与拾音单元，所述装置包括：
    检测单元，配置为检测所述屏幕发声单元的振动参数以得到振动信号，并根据所述振动信号生成第一参考信号；
    消除单元，配置为根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，所述第一音频信号为所述拾音单元拾取的音频信号，所述第一音频信号中至少包括第一回声信号，所述第一回声信号由所述屏幕发声单元驱动所述终端振动所产生的噪音形成。
  2. 根据权利要求1所述的装置，其中，所述检测单元包括：
    振动传感子单元，设置于所述屏幕发声单元的预设范围内，所述振动传感子单元配置为，检测所述屏幕发声单元的所述振动参数以得到所述振动信号；
    模数转换ADC子单元，配置为对所述振动信号进行模数转换处理，以得到所述第一参考信号。
  3. 根据权利要求1所述的装置，其中，所述消除单元配置为，
    对所述第一参考信号进行反相处理以生成第一滤波信号；
    将所述第一音频信号与所述第一滤波信号叠加，以对所述第一回声信号进行回声消除。
  4. 根据权利要求1至3任一项中所述的装置，其中，所述第一音频信号包括：所述第一回声信号、第二回声信号；
    其中，所述第二回声信号由所述屏幕发声单元根据第二音频信号驱动所述终端振动所输出的声音形成，所述第二音频信号为所述终端预输出的音频信号。
  5. 根据权利要求4所述的装置，其中，所述消除单元包括：
    第一消除子单元，配置为根据所述第一参考信号对所述第一回声信号进行回声消除；
    第二消除子单元，配置为根据第二参考信号对所述第二回声信号进行回声消除；其中，所述第二参考信号是根据所述第二音频信号生成的。
  6. 根据权利要求5所述的装置，其中，所述第一消除子单元还配置为，
    对所述第一参考信号进行反相处理以生成第一滤波信号；
    将所述第一音频信号与所述第一滤波信号叠加，以对所述第一回声信号进行回声消除。
  7. 根据权利要求5所述的装置，其中，所述第二消除子单元还配置为，
    通过自适应滤波器对所述第二参考信号进行滤波处理，以生成第二滤波信号；
    将所述第一音频信号与所述第二滤波信号叠加，以对所述第二回声信号进行回声消除。
  8. 根据权利要求1至3任一项中所述的装置，其中，所述拾音单元包括第一拾音单元与第二拾音单元；
    所述检测单元包括：
    第一检测子单元，设置于所述第一拾音单元的预设范围内，所述第一检测子单元配置为，检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号；
    第二检测子单元，设置于所述第二拾音单元的预设范围内，所述第二检测子单元配置为，检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
    所述装置还包括：
    处理单元，配置为根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频。
  9. 根据权利要求8所述的装置，其中，所述终端还包括第一屏幕发声单元与第二屏幕发声单元，其中，所述第一屏幕发声单元设置于所述第一拾音单元的预设范围内，所述第二屏幕发声单元设置于所述第二拾音单元的预设范围内；
    第一检测子单元，设置于所述第一拾音单元的预设范围内，所述第一检测子单元配置为，检测所述第一屏幕发声单元的振动参数以得到所述第一振动信号；
    第二检测子单元，设置于所述第二拾音单元的预设范围内，所述第二检测子单元配置为，检测所述第二屏幕发声单元的振动参数以得到所述第二振动信号。
  10. 根据权利要求8所述的装置，其中，所述处理单元还配置为，
    在所述第一振动信号所指示的振动频率小于所述第二振动信号所指示的振动频率的情形下，将所述第一拾音单元输入目标音频；或者，
    在所述第一振动信号所指示的振动频率大于所述第二振动信号所指示的振动频率的情形下，将所述第二拾音单元输入目标音频。
  11. 一种拾音装置，其中，设置于终端，所述终端至少包括屏幕发声单元与第一拾音单元、第二拾音单元，所述装置包括：
    检测单元，所述检测单元还包括：
    第一检测子单元，设置于所述第一拾音单元的预设范围内，所述第一检测子单元配置为，检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号；
    第二检测子单元，设置于所述第二拾音单元的预设范围内，所述第二检测子单元配置为，检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
    所述装置还包括：
    处理单元，配置为根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频。
  12. 根据权利要求11所述的装置，其中，所述终端还包括第一屏幕发声单元与第二屏幕 发声单元，其中，所述第一屏幕发声单元设置于所述第一拾音单元的预设范围内，所述第二屏幕发声单元设置于所述第二拾音单元的预设范围内；
    第一检测子单元，设置于所述第一拾音单元的预设范围内，所述第一检测子单元配置为，检测所述第一屏幕发声单元的振动参数以得到所述第一振动信号；
    第二检测子单元，设置于所述第二拾音单元的预设范围内，所述第二检测子单元配置为，检测所述第二屏幕发声单元的振动参数以得到所述第二振动信号。
  13. 根据权利要求11所述的装置，其中，所述处理单元还配置为，
    在所述第一振动信号所指示的振动频率小于所述第二振动信号所指示的振动频率的情形下，将所述第一拾音单元输入目标音频；或者，
    在所述第一振动信号所指示的振动频率大于所述第二振动信号所指示的振动频率的情形下，将所述第二拾音单元输入目标音频。
  14. 一种终端，包括权利要求1至10任一项中所述的回声消除装置，和/或，包括权利要求11至13任一项中所述的拾音装置。
  15. 一种回声消除方法，应用于终端，所述终端至少包括屏幕发声单元与拾音单元，所述方法包括：
    检测所述屏幕发声单元的振动参数以得到振动信号，并根据所述振动信号生成第一参考信号；
    根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除；其中，所述第一音频信号为所述拾音单元拾取的音频信号，所述第一音频信号中至少包括第一回声信号，所述第一回声信号由所述屏幕发声单元驱动所述终端振动所产生的噪音形成。
  16. 根据权利要求15所述的方法，其中，所述根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除，包括：
    对所述第一参考信号进行反相处理以生成第一滤波信号；
    将所述第一音频信号与所述第一滤波信号叠加，以对所述第一回声信号进行回声消除。
  17. 根据权利要求15或16所述的方法，其中，所述第一音频信号包括：所述第一回声信号、第二回声信号；
    其中，所述第二回声信号由所述屏幕发声单元根据第二音频信号驱动所述终端振动所输出的声音形成，所述第二音频信号为所述终端预输出的音频信号。
  18. 根据权利要求17所述的方法，其中，所述根据所述第一参考信号对第一音频信号中的第一回声信号进行回声消除，包括：
    根据所述第一参考信号对所述第一回声信号进行回声消除，并根据第二参考信号对所述第二回声信号进行回声消除；其中，所述第二参考信号是根据所述第二音频信号生成的。
  19. 根据权利要求18所述的方法，其中，所述根据所述第一参考信号对所述第一回声信 号进行回声消除，包括：
    对所述第一参考信号进行反相处理以生成第一滤波信号；
    将所述第一音频信号与所述第一滤波信号叠加，以对所述第一回声信号进行回声消除。
  20. 根据权利要求18所述的方法，其中，所述根据第二参考信号对所述第二回声信号进行回声消除，包括：
    通过自适应滤波器对所述第二参考信号进行滤波处理，以生成第二滤波信号；
    将所述第一音频信号与所述第二滤波信号叠加，以对所述第二回声信号进行回声消除。
  21. 根据权利要求15或16所述的方法，其中，所述终端包括第一拾音单元与第二拾音单元；
    所述方法还包括：
    检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号，并检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
    根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频。
  22. 根据权利要求21所述的方法，其中，所述终端还包括第一屏幕发声单元与第二屏幕发声单元，其中，所述第一屏幕发声单元设置于所述第一拾音单元的预设范围内，所述第二屏幕发声单元设置于所述第二拾音单元的预设范围内；
    所述检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号，并检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号，包括：
    检测所述第一屏幕发声单元的振动参数以得到所述第一振动信号，并检测所述第二屏幕发声单元的振动参数以得到所述第二振动信号。
  23. 根据权利要求21所述的方法，其中，所述根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频，包括：
    在所述第一振动信号所指示的振动频率小于所述第二振动信号所指示的振动频率的情形下，将所述第一拾音单元输入目标音频；或者，
    在所述第一振动信号所指示的振动频率大于所述第二振动信号所指示的振动频率的情形下，将所述第二拾音单元输入目标音频。
  24. 一种拾音方法，应用于终端，所述终端至少包括屏幕发声单元、第一拾音单元与第二拾音单元，所述方法还包括：
    检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号，并检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号；
    根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频。
  25. 根据权利要求24所述的方法，其中，所述终端还包括第一屏幕发声单元与第二屏幕发声单元，其中，所述第一屏幕发声单元设置于所述第一拾音单元的预设范围内，所述第二屏幕发声单元设置于所述第二拾音单元的预设范围内；
    所述检测所述终端中所述第一拾音单元所在区域的振动参数以得到第一振动信号，并检测所述终端中所述第二拾音单元所在区域的振动参数以得到第二振动信号，包括：
    检测所述第一屏幕发声单元的振动参数以得到所述第一振动信号，并检测所述第二屏幕发声单元的振动参数以得到所述第二振动信号。
  26. 根据权利要求24所述的方法，其中，所述根据所述第一振动信号与所述第二振动信号，选取所述第一拾音单元或所述第二拾音单元输入目标音频，包括：
    在所述第一振动信号所指示的振动频率小于所述第二振动信号所指示的振动频率的情形下，将所述第一拾音单元输入目标音频；或者，
    在所述第一振动信号所指示的振动频率大于所述第二振动信号所指示的振动频率的情形下，将所述第二拾音单元输入目标音频。
  27. 一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机程序，其中，所述计算机程序被设置为运行时执行所述权利要求15至23任一项中所述的方法，或者执行权利要求24至26任一项中所述的方法。
  28. 一种电子装置，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器被设置为运行所述计算机程序以执行所述权利要求15至23任一项中所述的方法，或者执行权利要求24至26任一项中所述的方法。
PCT/CN2021/094175 2020-06-24 2021-05-17 回声消除装置及方法、拾音装置及方法、终端 WO2021258913A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010589749.X 2020-06-24
CN202010589749.XA CN113840041A (zh) 2020-06-24 2020-06-24 回声消除装置及方法、拾音装置及方法、终端

Publications (1)

Publication Number Publication Date
WO2021258913A1 true WO2021258913A1 (zh) 2021-12-30

Family

ID=78964717

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/094175 WO2021258913A1 (zh) 2020-06-24 2021-05-17 回声消除装置及方法、拾音装置及方法、终端

Country Status (2)

Country Link
CN (1) CN113840041A (zh)
WO (1) WO2021258913A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116962583B (zh) * 2023-09-20 2023-12-08 腾讯科技（深圳）有限公司 一种回声控制的方法、装置、设备、存储介质及程序产品

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1795698A (zh) * 2003-05-28 2006-06-28 皇家飞利浦电子股份有限公司 显示屏扬声器
JP2012028870A (ja) * 2010-07-20 2012-02-09 Fujitsu Ltd 電気音響変換装置
US20120276961A1 (en) * 2011-04-26 2012-11-01 Kyocera Corporation Mobile communication device and echo cancellation method
WO2016146316A1 (en) * 2015-03-18 2016-09-22 Qualcomm Technologies International, Ltd. Structure for multi-microphone speech einhancement system
CN108683761A (zh) * 2018-05-17 2018-10-19 Oppo广东移动通信有限公司 发声控制方法、装置、电子装置及计算机可读介质
CN110018808A (zh) * 2018-12-25 2019-07-16 瑞声科技(新加坡)有限公司 一种音质调整方法及装置
CN110933557A (zh) * 2019-12-16 2020-03-27 歌尔股份有限公司 麦克风回音消除方法、装置、设备和计算机存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4281568B2 (ja) * 2003-07-31 2009-06-17 ソニー株式会社 通話装置
CN205992961U (zh) * 2016-01-29 2017-03-01 魏廉 可接收、显示多路门禁信号且带视频输出的门禁室内机
CN107592592A (zh) * 2017-07-28 2018-01-16 捷开通讯(深圳)有限公司 显示面板、移动终端及屏幕发声控制方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1795698A (zh) * 2003-05-28 2006-06-28 皇家飞利浦电子股份有限公司 显示屏扬声器
JP2012028870A (ja) * 2010-07-20 2012-02-09 Fujitsu Ltd 電気音響変換装置
US20120276961A1 (en) * 2011-04-26 2012-11-01 Kyocera Corporation Mobile communication device and echo cancellation method
WO2016146316A1 (en) * 2015-03-18 2016-09-22 Qualcomm Technologies International, Ltd. Structure for multi-microphone speech einhancement system
CN108683761A (zh) * 2018-05-17 2018-10-19 Oppo广东移动通信有限公司 发声控制方法、装置、电子装置及计算机可读介质
CN110018808A (zh) * 2018-12-25 2019-07-16 瑞声科技(新加坡)有限公司 一种音质调整方法及装置
CN110933557A (zh) * 2019-12-16 2020-03-27 歌尔股份有限公司 麦克风回音消除方法、装置、设备和计算机存储介质

Also Published As

Publication number Publication date
CN113840041A (zh) 2021-12-24

Similar Documents

Publication Publication Date Title
US9967661B1 (en) Multichannel acoustic echo cancellation
CN107636758B (zh) 声学回声消除系统和方法
US9653060B1 (en) Hybrid reference signal for acoustic echo cancellation
KR102196012B1 (ko) 트랜스듀서 상태의 검출에 기초하여 오디오 트랜스듀서의 성능을 향상시키는 방법들 및 시스템들
JP6092490B2 (ja) 広帯域周波数応答を提供するためのシステムおよび方法
CN105284127B (zh) 用于提供对于音频信号的频率响应的设备及方法
US10484792B2 (en) Headphone with noise cancellation of acoustic noise from tactile vibration driver
CN102099852A (zh) 噪声消除系统
TW201508735A (zh) 應用於音訊裝置的控制器與相關的操作方法
CN100518384C (zh) 一种无源电-声装置及其放音方法
JP2006139307A (ja) 声音効果処理と騒音制御を有する装置及びその方法
WO2015039441A1 (zh) 一种放音响度调整方法及装置
US9532125B2 (en) Noise cancellation microphones with shared back volume
KR20190126886A (ko) 사운드 변환기를 위한 증폭기 유닛 및 사운드 생성 유닛
CN103428593B (zh) 基于扬声器来采集音频信号的装置
CN111107462A (zh) 降噪系统，调试系统，方法，电子设备及存储介质
WO2021258913A1 (zh) 回声消除装置及方法、拾音装置及方法、终端
JPH09233198A (ja) 全二重音声会議電話のためのソフトウエアベースのブリッジ方法及び装置
GB2526945A (en) Noise cancellation microphones with shared back volume
TWI790718B (zh) 會議終端及用於會議的回音消除方法
WO2019119376A1 (en) Earphone and method for uplink cancellation of an earphone
JP5245884B2 (ja) 音声通信システム及び音声通信装置
CN104507033B (zh) 用于检测通信终端设备的电声换能器件的防尘性能的方法
WO2024087699A1 (zh) 音频增强方法、装置、计算设备集群及可读存储介质
TWI736122B (zh) 用於聲學回聲消除的時間延遲校準方法及電視裝置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21830203

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.05.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 21830203

Country of ref document: EP

Kind code of ref document: A1