WO2023061130A1 - Earphone, user device and signal processing method - Google Patents

Abstract

An earphone (2), a user device and a signal processing method. The earphone (2) comprises an audio input module (21), an ambient sound collection module (22), an active noise reduction module (23), an ambient sound processing module (24) and an audio playback module (25). The earphone (2) retains an ambient sound signal having a spatial sound effect in a designated pick-up direction, so as to avoid the problems caused by omnidirectional noise reduction methods, and to enable the ambient sound signal played back by the earphone (2) to sound more realistic.

Description

Headphone, user equipment and method for processing signals

priority information

This application requests the patent application number 202111190746.X submitted to the State Intellectual Property Office of China on October 12, 2021 and the patent application number 202111328416.2 submitted to the State Intellectual Property Office of China on November 10, 2021 Priority and Interest, which is hereby incorporated by reference in its entirety.

technical field

The present application relates to the technical field of earphones, and more specifically, to an earphone, user equipment and a signal processing method.

Background technique

With the development of random headphone technology, the application of headphones with active noise reduction function is becoming more and more widespread. The traditional active noise reduction method uses an omnidirectional noise reduction method, which eliminates environmental noise in all directions around the earphone wearer.

The advantage of the omnidirectional noise reduction method is that it can prevent the audio signal played by the earphone from being interfered by the environmental noise. However, in some occasions, the omnidirectional noise reduction method will cause problems such as safety. For example, when the earphone wearer is walking outdoors, if the earphone shields the sound of vehicles from behind, traffic accidents are likely to occur.

Contents of the invention

The present application provides an earphone, user equipment and a signal processing method to solve the above problems.

In a first aspect, a first earphone is provided, which includes: an audio input module configured to receive an audio signal to be played; an ambient sound collection module, which includes a first microphone and a second microphone configured to collect ambient sound signals around the first earphone Microphone; active noise reduction module, configured to generate a noise reduction signal according to the environmental sound signal; the environmental sound processing module, configured to obtain the first environmental sound signal of the specified pick-up direction from the environmental sound signal; the audio playback module, configured to play audio signal, the noise reduction signal and the mixing signal of the first ambient sound signal; wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the first direction and the length direction of the earphone handle of the first earphone are at an acute angle first angle.

In a second aspect, a wireless earphone is provided, which includes: a first audio receiver including a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module includes The first microphone and the second microphone of the ambient sound signal, the first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver to reduce the noise of the audio signal received by the first audio receiver The second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a fourth microphone configured to collect ambient sound signals around the second audio receiver , the second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce the noise of the audio signal received by the second audio receiver; wherein the first microphone and the second microphone are configured to Arranged along a first direction, the first direction forms a first angle that is an acute angle with the length direction of the handle of the first audio receiver, the third microphone and the fourth microphone are configured to be arranged along a second direction, and the second direction is aligned with the first The width direction of the handle of the audio receiver forms a second acute angle.

In a third aspect, a headset is provided, which includes: a first audio receiver, including a first environmental sound collection module and a first noise reduction module, wherein the first environmental sound collection module includes a receiver configured to collect the first audio The first microphone and the second microphone of the ambient sound signal around the receiver, and the first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver for the audio signal received by the first audio receiver Noise reduction; the second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a first microphone configured to collect ambient sound signals around the second audio receiver Four microphones, the second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce the noise of the audio signal received by the second audio receiver; wherein, the first microphone and the second microphone Configured to be arranged along a first direction, the third microphone and the fourth microphone are configured to be arranged along a second direction, wherein the first direction forms a first acute angle with the central axis of the headset, and the second direction forms a first angle with the central axis of the headset. The front of the earphone forms a second angle belonging to an acute angle.

The earphone provided in the embodiment of the present application does not adopt an omnidirectional noise reduction method, but a directional noise reduction method. That is to say, the earphone will retain the ambient sound signal in the specified pick-up direction, which can avoid problems such as safety caused by the omnidirectional noise reduction method. For example, the specified pickup direction can be set to backward to preserve the sound of vehicles behind the earphone wearer, thereby reducing the probability of traffic accidents. Furthermore, the ambient sound signal retained by the earphone also has a spatial sound effect, which enables the wearer of the earphone to locate the sound source of the sound signal, so that the ambient sound signal played by the earphone sounds more realistic.

Description of drawings

FIG. 1 is a schematic structural diagram of an earphone with an active noise reduction function.

Fig. 2 is a schematic structural diagram of an earphone provided by an embodiment of the present application.

Fig. 3 is an example diagram of a possible implementation manner of the environmental sound processing module in Fig. 2 .

Fig. 4 is an example diagram of the working principle of the beamformer.

FIG. 5 is an example diagram of a possible implementation of the directional sound pickup module in FIG. 3 .

FIG. 6 is an example diagram of a possible implementation manner of the sound effect processing module in FIG. 3 .

FIG. 7 is an example diagram of another possible implementation of the environmental sound processing module in FIG. 2 .

FIG. 8 is an example diagram of a possible implementation of the audio playing module in FIG. 2 .

FIG. 9 is a schematic flowchart of a customizing manner of a personalized HRTF filter provided by an embodiment of the present application.

FIG. 10 is an example diagram of another possible implementation manner of the sound effect processing module in FIG. 3 .

Fig. 11 is a schematic flowchart of a method for processing a signal provided by an embodiment of the present application.

Fig. 12A shows the position of the microphone array provided by one embodiment of the present application.

Fig. 12B shows a microphone array in the left earphone provided by an embodiment of the present application.

Fig. 12C shows a microphone array in the right earphone provided by an embodiment of the present application.

Fig. 12D shows a headset provided by an embodiment of the present application.

Detailed ways

The present application provides a first earphone, which includes:

an audio input module configured to receive an audio signal to be played;

An ambient sound collection module, which includes a first microphone and a second microphone configured to collect ambient sound signals around the first earphone;

An active noise reduction module configured to generate a noise reduction signal according to an ambient sound signal;

The environmental sound processing module is configured to obtain the first environmental sound signal of the specified pickup direction from the environmental sound signal;

The audio playback module is configured to play the mixing signal of the audio signal, the noise reduction signal and the first ambient sound signal;

Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the first direction forms a first acute angle with the length direction of the earphone stem of the first earphone.

Among them, the environmental sound processing module includes:

The directional sound pickup module is configured to obtain the picked-up ambient sound signal in a specified pickup direction from the ambient sound signal, and the picked-up ambient sound signal is a mono signal;

The sound effect processing module is configured to perform sound effect processing on the picked-up environmental sound signal to generate a first environmental sound signal with spatial sound effect.

Among them, the environmental sound processing module also includes:

The sound source direction estimation module is configured to: obtain the direction information of the sound source signal in the designated sound pickup direction;

The sound effect processing module is configured to: perform sound effect processing on the picked-up environmental sound signal according to the orientation information of the sound source signal to obtain a first environmental sound signal, wherein the first environmental sound signal has a spatial sound effect from the orientation of the sound source signal.

Among them, the sound effect processing module is configured as:

According to the orientation information of the sound source signal, determine the HRTF filter corresponding to the orientation of the sound source signal;

The HRTF filter is convolved with the picked-up ambient sound signal to obtain the first ambient sound signal.

Wherein, the ambient sound collection module is further configured to acquire ambient sound signals around the second earphone used in conjunction with the first earphone.

Wherein, the first direction includes the long axis direction of the main body of the first earphone.

Among them, the headset also includes:

The gain adjustment module is configured to adjust the intensity of the first ambient sound signal according to the instruction of the wearer of the earphone.

Wherein, the designated sound pickup direction includes the rear of the wearer of the earphone.

The present application provides a second earphone, which includes:

an audio input module configured to receive an audio signal to be played;

An ambient sound collection module, which includes a third microphone and a fourth microphone configured to collect ambient sound signals around the second earphone;

An active noise reduction module configured to generate a noise reduction signal according to an ambient sound signal;

The environmental sound processing module is configured to obtain the first environmental sound signal of the specified pickup direction from the environmental sound signal;

The audio playback module is configured to play the mixing signal of the audio signal, the noise reduction signal and the first ambient sound signal;

Wherein, the third microphone and the fourth microphone are configured to be arranged along a second direction, and the second direction forms a second acute angle with the width direction of the earphone handle of the second earphone.

Wherein, the ambient sound collection module is further configured to acquire ambient sound signals around the first earphone used in conjunction with the second earphone.

Wherein, the second direction includes the short axis direction of the main body of the second earphone.

The present application provides a wireless earphone, which includes the first earphone according to any one of claims 1-8 and the second earphone according to any one of claims 8-11.

The application provides a wireless earphone, which includes:

The first audio receiver includes a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module includes a first microphone and a second microphone configured to collect ambient sound signals around the first audio receiver, The first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver to reduce the noise of the audio signal received by the first audio receiver;

The second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a fourth microphone configured to collect ambient sound signals around the second audio receiver, The second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce the noise of the audio signal received by the second audio receiver;

Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the first direction forms a first angle belonging to an acute angle with the length direction of the handle of the first audio receiver, and the third microphone and the fourth microphone are configured to be arranged along The second direction is arranged, and the second direction forms a second angle belonging to an acute angle with the width direction of the handle of the first audio receiver.

Wherein, the first direction is the long axis direction of the main body of the first audio receiver, and the second direction is the short axis direction of the main body of the second audio receiver.

Wherein, the first direction is orthogonal to the second direction.

Wherein, the first environmental sound collection module is further configured to acquire the environmental sound signal around the second audio receiver, and the second environmental sound collection module is also configured to acquire the environmental sound signal around the first audio receiver.

Wherein, the first audio receiver also includes a first environmental sound processing module configured to obtain an environmental sound signal of a specified pick-up direction from the surrounding environmental sound signal, and the second audio receiver also includes a first environmental sound processing module configured to obtain the ambient sound signal from the surrounding environmental sound signal The second environmental sound processing module that obtains the environmental sound signal of the specified sound pickup direction in the middle.

Wherein, the first audio receiver further includes a first HRTF filter matched with the characteristics of the left ear of the wearer of the wireless earphone, and the second audio receiver further includes a second HRTF filter matched with the characteristic of the right ear of the wearer of the wireless earphone. HRTF filter.

Wherein, the designated sound pickup direction includes the rear of the wearer of the wireless earphone.

The application provides a headset, which includes:

The first audio receiver includes a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module includes a first microphone and a second microphone configured to collect ambient sound signals around the first audio receiver, The first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver to reduce the noise of the audio signal received by the first audio receiver;

The second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a fourth microphone configured to collect ambient sound signals around the second audio receiver, The second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce the noise of the audio signal received by the second audio receiver;

Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the third microphone and the fourth microphone are configured to be arranged along a second direction, wherein the first direction forms a first acute angle with the central axis of the headset. Angle, the second direction forms a second angle that is an acute angle with the front of the headset.

Wherein, the first direction is orthogonal to the second direction.

Wherein, the first environmental sound collection module is further configured to acquire the environmental sound signal around the second audio receiver, and the second environmental sound collection module is also configured to acquire the environmental sound signal around the first audio receiver.

Wherein, the first audio receiver also includes a first environmental sound processing module configured to obtain an environmental sound signal of a specified pick-up direction from the surrounding environmental sound signal, and the second audio receiver also includes a first environmental sound processing module configured to obtain the ambient sound signal from the surrounding environmental sound signal The second environmental sound processing module that obtains the environmental sound signal of the specified sound pickup direction in the middle.

The present application provides a user equipment, which includes the wireless headset according to any one of claims 13-20 or the headset according to any one of claims 21-23.

The present application provides an audio system, which includes a sound source, and the wireless earphone according to any one of claims 13-19 or the headset according to any one of claims 20-23.

The present application provides a method for processing an audio signal, wherein the method is applied to a first earphone, and the method includes the following steps:

S1. Receive an input audio signal;

S2. Utilize the first microphone and the second microphone in the first earphone to collect the ambient sound signal around the first earphone; wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the first direction is substantially parallel to the wearing the direct rear direction of the first earphone or substantially parallel to the vertical direction of the first earphone;

S3. Generate a first noise reduction signal based on the environmental sound signal;

S4. Acquiring the first environmental sound signal of the specified pickup direction from the environmental sound signal; and

S5. Outputting an audio signal to be played based on the input audio signal, the first noise reduction signal, and the first ambient sound signal.

Wherein, step S4 includes:

Obtaining a pickup ambient sound signal in a specified pickup direction from the ambient sound signal, wherein the pickup ambient sound signal is a mono signal;

Sound effect processing is performed on the picked-up ambient sound signal to generate a first ambient sound signal with spatial sound effect.

Among them, the method also includes:

Obtain the orientation information of the sound source signal in the specified pick-up direction;

The sound effect processing is performed on the picked-up environmental sound signal according to the orientation information of the sound source signal to obtain a first environmental sound signal, wherein the first environmental sound signal has a spatial sound effect from the orientation of the sound source signal.

Among them, the method also includes:

According to the orientation information of the sound source signal, determine the HRTF filter corresponding to the orientation of the sound source signal;

The HRTF filter is convolved with the picked-up ambient sound signal to obtain the first ambient sound signal.

Among them, the method also includes:

obtaining an image of the ear of the wearer of the headset;

extract ear features from ear images;

Select an HRTF filter that matches the ear features.

Among them, the method also includes:

Under the premise that the signal integrity index of the environmental sound processing module is equal to the number of microphones in the first earphone and the signal directivity index of the environmental sound processing module is equal to the square of the number of microphones, the environment is determined based on making the energy of the first environmental sound signal the minimum The weighting matrix of the sound processing module, wherein the environmental sound processing module is configured to obtain the first environmental sound signal from the environmental sound signal.

Among them, the method also includes:

The intensity of the first ambient sound signal is adjusted according to the instruction of the wearer of the earphone.

The present application provides a method for processing an audio signal, which is applied to a wireless stereo headset, wherein the method includes:

S1. Using the first earphone and the second earphone to receive the first audio signal and the second audio signal respectively;

S2. Utilize the first microphone and the second microphone in the first earphone to collect the ambient sound signal around the first earphone, and use the third microphone and the fourth microphone in the second earphone to collect the ambient sound signal around the second earphone;

Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the third microphone and the fourth microphone are configured to be arranged along a second direction, wherein, the first direction is substantially parallel to the direction directly behind the wearer, and the second microphone is configured to be arranged along a second direction. The two directions are substantially parallel to the vertical direction of the second earphone;

S3. Generate a first noise reduction signal and a second noise reduction signal based on the environmental sound signal around the first earphone and the environmental sound signal around the second earphone;

S4. Obtain the first environmental sound signal of the designated sound pickup direction from the environmental sound signal around the first earphone, and obtain the second environmental sound signal of the designated sound pickup direction from the environmental sound signal around the second earphone; and

S5. Generate a mixed signal of the first audio signal, the first noise reduction signal, and the first ambient sound signal, and generate a mixed signal of the second audio signal, the second noise reduction signal, and the second ambient sound signal.

Among them, the method also includes:

Obtain the orientation information of the sound source signal in the specified pick-up direction;

determining a first angular change of the sound source signal relative to a direction directly behind the wearer and a second angular change relative to a vertical direction of the second earphone;

An attenuation matrix of the microphone array is determined based on the first angle change and the second angle change, wherein the microphone array includes a first microphone, a second microphone, a third microphone, and a fourth microphone.

Among them, the method also includes:

Under the premise that the signal integrity index of the environmental sound processing module is equal to the number of microphones in the microphone array and the signal directivity index of the environmental sound processing module is equal to the square of the number of microphones, based on making the energy of the first environmental sound signal the minimum and the second The energy of the environmental sound signal is minimized to determine the weighting matrix of the environmental sound processing module, wherein the environmental sound processing module is configured to obtain the first environmental sound signal and/or the second environmental sound signal from the environmental sound signal.

Further description will be made below in conjunction with the accompanying drawings.

With the advancement of technology, the performance of headphones with noise reduction function on the market is getting better and better. At present, there are two main noise reduction technologies for headphones: passive noise reduction technology and active noise cancellation technology (active noise cancellation, referred to as ANC).

Passive noise cancellation technology has been around for a long time. Passive noise reduction technology mainly surrounds the ear with earphones to form a closed space to block external noise. Alternatively, the earphones can also use sound-insulating materials such as silicone earplugs to block external noise, thereby achieving passive noise reduction.

Active noise reduction technology refers to the generation of noise reduction signals corresponding to the environmental noise around the earphones through the active noise reduction module, thereby reducing or even eliminating the environmental noise to achieve the noise reduction effect. The noise reduction signal can be, for example, a sound signal that is equal in amplitude to the external noise but opposite in phase. In the past, due to reasons such as size, power consumption, and cost, headphones with active noise reduction functions usually only existed in certain special industrial fields. In recent years, with the advancement of technology and the improvement of consumption levels, active noise reduction technology has been more and more widely used in the field of consumer electronics, making headphones with active noise reduction functions more and more popular among consumer groups. higher.

The active noise reduction technology currently used in headphones is mainly divided into three noise reduction modes, namely feed-forward noise reduction, feedback noise reduction and hybrid noise reduction. There are certain differences in the acoustic structure and signal processing methods of earphones corresponding to different noise reduction modes. Therefore, different noise reduction modes have their own characteristics in terms of noise reduction depth and noise reduction bandwidth.

A noise reduction module adopting a feedforward noise reduction mode may be called a feedforward noise reduction module. As shown in (a) of FIG. 1 , the earphone can receive an externally input audio signal. After the audio signal is processed by the active noise reduction circuit, it can be played through the loudspeaker for the earphone wearer to listen to. The feedforward noise reduction module may include a feedforward microphone (or called a feedforward microphone) and an active noise reduction circuit shown in (a) of FIG. 1 . The feed-forward microphone can be used to detect the noise signal of the surrounding environment of the earphone, and output a signal (or anti-phase signal) with the same frequency response as the environmental noise of the earphone (or anti-phase signal) through the active noise reduction circuit to realize the active noise reduction function. At the eardrum of the headphone wearer, this anti-phase signal cancels out the noise signal, thereby reducing the noise level heard by the human ear. Due to signal transmission delay and other reasons, there is a certain difference between the noise signal detected at the feedforward microphone and the noise signal at the ear drum, and the active noise reduction circuit needs to compensate for this difference.

A noise reduction module adopting a feedback noise reduction mode may be referred to as a feedback noise reduction module. Referring to (b) in FIG. 1 , the main difference from the feedforward noise reduction module shown in (a) in FIG. 1 is that the feedback noise reduction module replaces the feedforward microphone with the feedback microphone. The feedback noise reduction module mainly uses the feedback microphone to detect the noise signal in the eardrum area, and then forms a feedback path to minimize the noise level in this area.

A noise reduction module using a hybrid noise reduction mode may be referred to as a hybrid noise reduction module. Referring to (c) in FIG. 1 , the hybrid noise reduction module includes both a feedforward microphone and a feedback microphone. In the hybrid noise reduction module, the noise reduction signal emitted by the speaker of the earphone is jointly determined by the feedforward microphone and the feedback microphone. The feed-forward noise reduction module in the hybrid noise reduction module can reduce the high-frequency noise signal in the noise signal of the surrounding environment of the earphone, and the feedback noise reduction module can reduce the low-frequency noise signal in the noise signal. The cooperation between the feed-forward noise reduction module and the feedback noise reduction module can effectively enhance the flexibility of the noise reduction module, and the noise reduction effect is also very effective.

Headphones with active noise reduction on the market achieve omnidirectional noise reduction. That is to say, noise signals from all directions around the earphone will be suppressed by the noise reduction module of the earphone. The advantage of the omnidirectional noise reduction method is that it can ensure that the audio signal played in the earphone is almost completely free from noise signal interference. However, in some occasions, the omnidirectional noise reduction method may cause problems in user experience, and may even cause certain security risks.

For example, when the wearer of the headset wishes to communicate with a friend on the left side while listening to music, it is difficult to hear the friend unless the wearer of the headset takes off the headset because the voice of the friend will be canceled by the noise reduction module. the sound of. In this case, if the ambient sound signal on the left side of the earphone can be preserved, the user experience of the earphone wearer can be improved.

As another example, when the earphone wearer listens to music in public places, especially on the road, the sound from the vehicle behind the body will be eliminated by the noise reduction module in the earphone, which will cause the earphone wearer to be unable to feel the surrounding environment. Potential risk, it is easy to cause traffic accidents. Therefore, in this case, it is necessary to keep the ambient sound signal outside the line of sight (especially behind the earphone wearer) while suppressing the ambient sound signal within the line of sight (in front of the earphone wearer), so as to Improve the safety of users when traveling with headphones.

In order to solve the above problem, the structure of the earphone provided by the embodiment of the present application will be described in detail below with reference to FIG. 2 .

As shown in FIG. 2 , the earphone 2 may include an audio input module 21 , an ambient sound collection module 22 , an active noise reduction module 23 , an ambient sound processing module 24 and an audio playback module 25 .

The audio input module 21 (or audio input circuit) can be used to receive the audio signal to be played. The audio input circuit 21 may include, for example, one or more audio signal interfaces. Through the one or more audio data interfaces, one or more types of audio signals can be input into the earphone 2 . The audio signal received by the audio input module 21 may be, for example, a music signal or a sound signal.

The ambient sound collection module 22 can be used to collect ambient sound signals around the earphone 2 . The ambient sound collection module 22 may include, for example, a plurality of microphones, and the plurality of microphones may include, for example, a feedforward microphone, a feedback microphone, a call microphone, and other auxiliary microphones. The plurality of microphones may be located at different locations on the headset. For example, the multiple microphones may be distributed in the left and right earphone units of the earphone. The multiple microphones may form a microphone array (microphone array, MA for short).

The environmental sound signal output by the environmental sound collection module 22 may be an omnidirectional environmental sound signal. That is to say, the ambient sound signal output by the ambient sound collection module 22 may include sound signals from various directions around the earphone 2 . Since the ambient sound signal collected by the feedforward microphone and the feedback microphone is generally considered to be the noise signal corresponding to the audio signal played by the earphone, in some embodiments, the ambient audio signal collected by the feedforward microphone and the feedback microphone can also be It is called the ambient noise signal.

The active noise reduction module 23 can be used to generate a noise reduction signal according to the environmental sound signal output by the environmental sound collection module 22 . The input terminal of the active noise reduction module 23 can be electrically connected with the output terminal of the environmental sound collection module 22 to receive the environmental noise signal from the output terminal of the environmental sound collection module 22 . The output terminal of the active noise reduction module 23 may output a noise reduction signal corresponding to the environmental noise signal.

The noise reduction signal corresponding to the environmental noise signal can be used to reduce or cancel the environmental noise signal. For example, the noise reduction signal can be an inverse signal of the ambient noise signal. In other words, the noise reduction signal may have the same frequency response as the ambient noise signal but an opposite phase. In some embodiments, the noise reduction signal output by the active noise reduction module 23 may not be the final noise reduction signal, but needs to go through some processing steps before being used for noise reduction.

The active noise reduction module 23 may adopt any one of the feedforward noise reduction module, the feedback noise reduction module, and the hybrid noise reduction module shown in FIG. 1 . The feed-forward noise reduction module has the advantage of covering a wide range of noise reduction, but it is difficult to fine-tune the noise reduction effect. The noise reduction range of the feedback noise reduction module is relatively small, but it can make fine adjustments to low-frequency signals. The hybrid noise reduction module has the performance of both the feedforward noise reduction module and the feedback noise reduction module, but the power consumption and cost of the hybrid noise reduction module may be higher. Therefore, an appropriate active noise reduction mode can be set for the active noise reduction module 23 according to the characteristics of various noise reduction modules and the actual needs of users.

The environmental sound processing module 24 may be configured to obtain the first environmental sound signal of a designated sound pickup direction from the environmental sound signals output by the environmental sound collection module 22 . The input terminal of the environmental sound processing module 24 can be electrically connected with the output terminal of the environmental sound collection module 22 to receive the environmental sound signal output by the environmental sound collection module 22 from the environmental sound collection module 22 . In addition, the output terminal of the environmental sound processing module 24 can be used to output the first environmental sound signal.

In some embodiments, the first audio receiver (such as the left earphone) includes a first environmental sound processing module configured to obtain an environmental sound signal of a designated sound pickup direction (such as directly behind the wearer) from the surrounding environmental sound signals , similarly, the second audio receiver (such as the right earphone) includes a second ambient sound processing module configured to obtain an ambient sound signal in a designated pickup direction (such as directly behind the wearer) from the ambient sound signal around it.

In some embodiments, the ambient sound collection module 22 in the first earphone not only collects the ambient sound signal around the first earphone (eg, the left earphone), but also acquires the environmental signal around the second earphone (eg, the right earphone). Similarly, the ambient sound collection module in the second earphone not only collects the ambient sound signal around the second earphone (eg, the right earphone), but also acquires the environmental signal around the first earphone (eg, the left earphone). In this way, the ambient sound from the designated sound pickup direction can be fully preserved, thereby not only enhancing the effect of active noise reduction, but also enabling the wearer of the headset to perceive the movement of rear objects or objects in a timely manner.

The first ambient sound signal is a signal with spatial sound effects. Spatial sound effects can also be referred to as stereo sound effects. Therefore, the first ambient sound signal may also be referred to as a stereo signal or a multi-channel signal. The embodiment of the present application does not specifically limit the number of channel signals included in the first environmental sound signal. For example, the first ambient sound signal may be a binaural signal. In another example, the first ambient sound signal may be a five-channel signal with surround sound.

"Specify" in specifying the pick-up direction can be understood as a preset. The pick-up direction can be specified when the headset leaves the factory. Alternatively, the sound pickup direction can also be specified and/or adjusted by the earphone wearer according to actual requirements. For example, in the process of listening to music, if the earphone wearer wishes to hear sounds from the rear to avoid traffic hazards, the earphone wearer can set the designated sound pickup direction as "rear". As another example, in the process of listening to music, if the earphone wearer wishes to talk to a friend on the left side, the earphone wearer can set the designated sound pickup direction as "left".

The specified pick-up direction can refer to the range of directions. For example, if the designated sound pickup direction is backward, the designated sound pickup direction may refer to a range of 60 degrees to the left and right directly behind the earphone wearer, or a range of 90 degrees to the left and right directly behind the earphone wearer.

As mentioned above, the first environmental sound signal output by the environmental sound processing module 24 is a signal with spatial sound effects. The embodiment of the present application does not specifically limit the manner of generating the first environmental sound signal. As an example, the environmental sound processing module 24 may perform filtering processing on the left channel signal collected by the environmental sound collection module 22, so as to extract the environmental sound signal a of a specified sound pickup direction from the left channel signal. Further, the environmental sound processing module 24 may also perform filtering processing on the right channel signal collected by the environmental sound collection module 22, so as to extract the environmental sound signal b of the designated sound pickup direction from the right channel signal. The environmental sound signal a and the environmental sound signal b can form a binaural signal with a spatial sound effect in a designated sound pickup direction.

As another example, the environmental sound processing module 24 may perform directional enhancement on the environmental sound signal output by the environmental sound collection module 22 based on the beamforming filter, so as to obtain the picked-up environmental sound signal. Since the beamforming filter weights and synthesizes signals of multiple channels into a signal of one channel, the sound pickup environment signal is a mono signal. Then, the ambient sound processing module 24 may perform sound rendering on the picked-up ambient sound signal to obtain a first ambient sound signal with spatial sound effects. This implementation manner will be described in detail later in conjunction with FIG. 3 , and will not be described in detail here.

The audio playing module 25 can be used to play the mixed signal of the audio signal, the noise reduction signal and the first ambient sound signal. The audio playing module 25 may include a speaker, for example. In some embodiments, the audio playing module 25 may also include a sound mixing module. Refer to FIG. 8 for a detailed description of the implementation of the audio playback module 25 .

The earphone with the active noise reduction function provided by the embodiment of the present application does not perform omnidirectional noise reduction on the environmental sound signal, but retains the sound signal in a specified pick-up direction. The preservation of the sound signal in the specified pick-up direction can improve the avoidance of a series of problems caused by the omni-directional noise reduction mentioned above. For example, in an outdoor scene, the designated sound pickup direction can be set to the rear of the earphone wearer, so that the earphone wearer can hear the sound of vehicles behind, thereby improving the safety of wearing the earphone. For another example, in the scenario of chatting while listening to music, the specified pick-up direction can be set to the direction where the chatting object is located, so that the earphone wearer can listen to music and chat with friends at the same time. In addition, in the embodiment of the present application, the ambient sound signal played by the earphone has a spatial sound effect, so that the wearer of the earphone can locate the direction of the sound signal, so that the ambient sound signal sounds more real.

A possible implementation of the environmental sound processing module 24 is given below in conjunction with FIG. 3 .

Referring to FIG. 3 , the environmental sound processing module 24 may include a directional sound pickup module 241 . The directional sound pickup module 241 may perform directional sound pickup based on a beamforming filter. The beamforming filter uses the principle of beamforming to perform directional enhancement on multiple signals in space, and finally forms a directional enhanced signal. The signal output by the beamforming filter has high signal quality. In the embodiment of the present application, the beamforming filter can be used to process the environmental sound signal output by the environmental sound collection module 22, so as to obtain the environmental sound signal of the designated sound pickup direction. For the convenience of description, the signal output by the directional sound pickup module 241 will be referred to as a sound pickup ambient sound signal hereinafter.

Referring to FIG. 4 , the signal shown in FIG. 4 is the environmental sound signal output by the environmental sound collection module 22 . The environmental sound signal output by the environmental sound collection module 22 may include signals of various frequency points (100 Hz, 500 Hz, 1000 Hz, 5000 Hz as shown in FIG. 4 ). The forward direction in Fig. 4 represents the front of the body of the earphone wearer; the rear direction represents the rear of the body of the earphone wearer. The beamforming filter will set a smaller gain for the forward signal of each frequency point in the ambient sound signal output by the ambient sound acquisition module 22, and set a larger gain for the backward signal, so that the earphone wearer's rearward signal can be effectively enhanced. The ambient sound signal of the headset suppresses the ambient sound signal located in front of the earphone wearer, thereby forming the effect of directional sound pickup.

As mentioned above, the environmental sound collection module 22 may use multiple microphone arrays to collect environmental sound signals around the earphone. As shown in FIG. 5 , the microphone array may include a left-ear microphone 221 and a right-ear microphone 222 . Since the left-ear microphone 221 and the right-ear microphone 222 are respectively located in the left and right earphone units of the earphone, there will be differences in ambient sound signals collected by the left-ear microphone 221 and the right-ear microphone 222 . Therefore, in some embodiments, before using the directional sound pickup module 241 to perform beamforming 2412 , the environmental sound signals collected by the left-ear microphone 221 and the right-ear microphone 222 may also be synchronously processed 2411 . For example, the signals collected by the left-ear microphone 221 and the right-ear microphone 222 can be transmitted to each other through wired or wireless means, and then the signals are synchronized on their respective sides. According to the principle of auditory psychology, the time delay between the signals collected by the left-ear microphone 221 and the right-ear microphone 222 needs to be controlled within 20 ms, otherwise the effect of subsequent sound mixing will be affected.

Referring again to FIG. 3 , the directional sound pickup module 241 converts the ambient sound signal output by the ambient sound collection module 22 into a picked-up ambient sound signal based on a beamforming filter. Since the working principle of the beamforming filter is to synthesize multiple signals into one signal, the picked-up ambient sound signal is a mono signal. In order to improve the auditory effect of the ambient sound signal, the sound effect processing module 242 can be used to perform sound effect processing (or sound effect rendering) on the picked-up ambient sound signal, and convert it into a first environmental sound signal with spatial sound effects, so as to enhance the earphone wearer. auditory experience.

In some embodiments, as shown in FIG. 6 , the sound effect processing module 242 may be implemented based on a head related transfer function (head related transfer function, HRTF for short) filter 2421.

The HRTF filter 2421 can be understood as a sound effect rendering algorithm. Specifically, people have two ears, but they can locate sounds from three-dimensional space, which is beneficial to the ability of human ears to analyze sound signals. The digital representation of this analytical capability is the HRTF filter 2421 . That is to say, the sound source signal transmitted from any point in space to the human ear (in front of the eardrum) can be described by one HRTF filter 2421 . The sound source signal passes through the HRTF filter 2421 to obtain the sound signal in front of the eardrums of both ears. HRTF filter 2421 can be regarded as a black box. After the HRTF filter 2421 describing the transmission relationship between the sound source signal of a certain orientation in space and the sound signal in front of the eardrums of both ears is obtained in a certain way, the sound source of the orientation can be restored based on the HRTF filter 2421 signal, resulting in spatial sound effects.

HRTF filter 2421 can be fitted by a mathematical function. Alternatively, the HRTF filter 2421 can also be measured through experiments. As an example, the HRTF filter library collected by laboratory tests can be stored in the headset, which can be called by the headset during actual work.

The process of implementing spatial sound effects based on the HRTF filter is illustrated below with an example. Referring to FIG. 6 , the orientation information of the sound source signal may be acquired first. Then, select the HRTF filter (or the parameters of the HRTF filter) corresponding to the direction of the sound source signal from the HRTF filter 2421 . Next, the convolution module 2422 can be used to convolve the sound pickup ambient sound signal output by the directional sound pickup module 241 (see FIG. 3 ) with the HRTF filter corresponding to the direction of the sound source signal, so as to obtain a spatial sound effect The first environmental sound signal.

Exemplarily, it is assumed that h _l (n) and h _r (n) respectively represent the impulse responses of the HRTF filters of the left ear and the right ear in a certain direction (or angle), and x(n) represents the first environmental sound signal , then after being filtered by the HRTF filter, the output signal is:

In the above formula, y _l (n) and y _r (n) represent the HRTF impulse response output of the left ear and the right ear respectively. After obtaining y _l (n) and y _r (n), it is equivalent to obtaining Two-channel sound signal for spatial sound effects.

Before utilizing the sound effect processing module 242 to process, the orientation information of the sound source signal (which can be the direction information of the sound source signal or the angle information of the sound source signal) can be determined earlier, and then the directional sound pickup module 241 outputs The picked-up ambient sound signal is rendered into a first ambient sound signal, so that the first ambient sound signal has a spatial sound effect from the position of the sound source signal. A possible way of determining the orientation information of the sound source signal is given below with reference to FIG. 7 .

As shown in FIG. 7 , in some embodiments, the environmental sound processing module 24 may further include a sound source direction estimation module 243 . The sound source direction estimating module 243 may be used to acquire the direction information of the sound source signal in the specified sound pickup direction. As mentioned above, the specified pick-up direction can be a range of directions, therefore, the sound source orientation estimation module 243 can obtain the accurate orientation of each sound source signal within the range of directions.

In some embodiments, the sound source azimuth estimating module 243 can use the direction of arrival (Direction of Arrival, DOA) to estimate the azimuth information of the sound source signal. Taking the environmental sound collection module 22 as a microphone array as an example, multiple microphones in different positions in the microphone array can be used to collect the arrival direction of the sound source signal, and then the orientation information of the sound source signal in the specified sound pickup direction can be obtained based on the DOA principle.

In some embodiments, as shown in FIG. 8 , the audio playing module 25 may include a sound mixing module 251 and a speaker 252 . The mixing module 251 can be used to mix the audio signal, the noise reduction signal and the first ambient sound signal, so as to generate a signal to drive the speaker 252 to produce sound.

In some embodiments, since the low-frequency part of the audio signal will be attenuated during the sound mixing process, as shown in FIG. Loss of the low frequency portion of an audio signal.

In some embodiments, as shown in FIG. 8 , before mixing, the phase inversion module 254 can be used to invert and adjust the noise reduction signal output by the noise reduction module, so that it can be used to cancel the environmental noise.

In some embodiments, in order to allow the earphone wearer to flexibly adjust the hearing experience of the audio signal (such as a music signal) and the sensitivity to the rear sound source, as shown in FIG. 8 , a gain adjustment module 255 can be set in the audio playback module 25 , to flexibly adjust the strength of the sound source signal in the specified pick-up direction. Taking the designated sound pick-up direction as the rear as an example, the strength of the rear sound signal (such as a sound signal from a vehicle) can be adjusted through the gain adjustment module 255, thereby controlling the safety level.

As mentioned above, the sound effect processing module 242 can be implemented based on the HRTF filter. Due to the large difference between the general HRTF and the real individual situation of the user, the effect of the general HRTF filter library is generally not good. Thus, in some embodiments, a personalized HRTF filter may be customized for the headset wearer. In the following, with reference to FIG. 9 and FIG. 10 , an example of the personalized customization method of the HRTF filter will be described in detail.

Referring to Fig. 9 and Fig. 10, firstly, the ear image of the wearer of the earphone can be acquired. Then, feature detection (landmark detection) is performed on the ear image to extract ear features of the earphone wearer from the ear image. Next, an HRTF filter that matches (coarsely matches) the ear features may be selected from the HRTF filter database. Then, the parameters of the HRTF filter can be corrected or fine-tuned based on the ear size of the earphone wearer and the head and torso (HAT), so as to obtain a personalized HRTF filter. The processes described in Figs. 9 and 10 may be performed when the earphone is not in use. For example, after the earphone wearer purchases the earphone, he can use the application software supporting the earphone to perform the above process to customize a personalized HRTF filter. After getting the personalized HRTF filter, and then enable the earphone to play the audio signal, the earphone can use the personalized HRTF filter to achieve accurate sound effect rendering.

The embodiment of the present application also provides a user equipment, where the user equipment includes a wireless communication unit and an earphone, where the earphone can be any earphone described above.

The device embodiment of the present application is described in detail above with reference to FIG. 1 to FIG. 10 , and the method embodiment of the present application is described in detail below with reference to FIG. 11 . It should be understood that the descriptions of the method embodiments correspond to the descriptions of the device embodiments, therefore, for parts not described in detail, reference may be made to the foregoing device embodiments.

Fig. 11 is a schematic flowchart of a method for processing a signal provided by an embodiment of the present application. The method in Fig. 11 can be applied to the aforementioned earphones. The method in FIG. 11 may include steps S1110 to S1150.

In step S1110, an audio signal to be played is received.

In step S1120, the ambient sound signal around the earphone is collected.

In step S1130, a noise reduction signal is generated according to the environmental sound signal.

In step S1140, a first environmental sound signal specifying a sound pickup direction is obtained from the environmental sound signal. The first ambient sound signal is a multi-channel signal with spatial sound effects.

In step S1150, the mixed signal of the audio signal, the noise reduction signal and the first ambient sound signal is played.

Optionally, step S1140 may include: acquiring a sound-picking environmental sound signal of a designated sound-picking direction from the environmental sound signal, where the sound-picking environmental sound signal is a monophonic signal; performing sound effect processing on the sound-picking environmental sound signal to generate The first ambient sound signal of the spatial sound effect.

Optionally, the method in FIG. 11 may also include: acquiring the orientation information of the sound source signal specifying the sound pickup direction; step S1140 may include: performing sound effect processing on the sound pickup environmental sound signal according to the orientation information of the sound source signal, to obtain the first An environmental sound signal, so that the first environmental sound signal has a spatial sound effect from the direction of the sound source signal.

Optionally, performing sound effect processing on the picked-up ambient sound signal according to the orientation information of the sound source signal to obtain the first environmental sound signal may include: determining an HRTF filter corresponding to the orientation of the sound source signal according to the orientation information of the sound source signal; The HRTF filter is convoluted with the picked-up ambient sound signal to obtain the first ambient sound signal.

Optionally, before receiving the audio signal to be played, the method in FIG. 11 may also include: acquiring an ear image of the wearer of the earphone; extracting ear features from the ear image; selecting an ear feature from the HRTF filter database. HRTF filter with internal feature matching.

Optionally, the environmental sound signal is collected by multiple microphones, and obtaining the orientation information of the sound source signal in the specified sound pickup direction may include: obtaining the sound source in the specified sound pickup direction according to the arrival directions of the sound source signals collected by the multiple microphones The location information of the signal.

Optionally, the method in FIG. 11 may further include: adjusting the intensity of the first ambient sound signal according to the instruction of the wearer of the earphone. Optionally, the designated sound pickup direction is behind the earphone wearer.

12A-12D show earphones (including wireless earphones and headsets) provided by some embodiments of the present application, in which a microphone array with a specific structure is used.

As shown in Figure 12A, in one embodiment, a microphone array composed of four microphones is arranged in two earphones, and multiple microphones M1, M2, M3, and M4 of the microphone array are distributed in the left and right earphone monomers of the earphone , where two microphones M1 and M2 are placed in the left earphone, and two microphones M3 and M4 are placed in the right earphone. Fig. 12A shows three-dimensional coordinate axes, the X-axis points to the back of the wearer's body, the Y-axis points to the right of the wearer, and the Z-axis points to the vertical upward direction.

Continuing to refer to FIG. 12A, the first and second microphones M1 and M2 are arranged along a first direction, wherein the first direction is substantially parallel to the Z-axis direction, and the third and fourth microphones M3 and M4 are arranged along a second direction, wherein the second The direction is substantially parallel to the X-axis direction. In this way, there is an angle close to 90 degrees between the arrangement direction of the first and second microphones and the arrangement direction of the third and fourth microphones. It should be understood that the included angle is not limited to the example of 90 degrees. In some embodiments, there is a first included angle which is an acute angle between the first direction and the Z-axis direction. In some other embodiments, there is also a second included angle that is an acute angle between the second direction and the X-axis, and the second included angle may be different from the first included angle.

More specifically, with the center of the headphone wearer's brain as the origin, the first and second microphones M1 and M2 are respectively placed at three-dimensional coordinates (0,-9,2), (0,-9,-2 ) position (located inside the first earphone), and then place the third and fourth microphones M3 and M4 respectively at the positions of three-dimensional coordinates (2,9,0), (-2,9,0) (located in the first two earphones). For the sake of illustration, the first earphone is the left earphone, the second earphone is the right earphone, and the sound source that the earphone wearer needs to pay attention to comes from the rear as an example.

In the case where the sound source comes from directly behind, the sound source direction is defined as (α=90°, θ=0°), where α represents the horizontal angle of the sound source direction (equal to the angle between the Y-axis and the Z-axis plane), θ represents the pitch angle of the sound source direction (equal to the included angle with the X-axis and Z-axis plane). The microphone array collects the sound signal directly behind the earphone. When the sound source moves, the X-axis pointing directly behind the wearer's body and the Y-axis pointing to the right of the wearer will also change accordingly, and then α, θ will follow the sound. The movement of the source changes, thereby changing the pointing direction of the microphone array relative to the coordinate system.

According to an improved embodiment, in the directivity index

and signal integrity metrics

Under the constraints of , in order to make the sound source signal perceivable by the wearer, it is hoped that the overall energy of the processed ambient sound signal is the minimum min _h(w) P(w)=h ^H (w)*[p*I _m +T _α,θ (w)]*h(w),

This results in the following calculation formula:

Where w is the frequency of the signal, α is the horizontal angle, θ is the pitch angle, h(w) is the weighting matrix of the environmental sound processing module, d(w, α, θ) is the correlation delay matrix of the microphone array, T _{α, θ} (w) is the pseudo-correlation matrix between the microphone arrays, and M is the number of microphones in the microphone array. In this way, by balancing the signal integrity and beam directivity to obtain the beamforming of the desired target direction, the technical effect of the beam direction following the movement of the object is achieved when an object passes behind.

As shown in FIG. 12B , according to an embodiment of the present application, the first earphone 31 includes a main body and an earphone stem. The length of the earphone stem is L, the width is W, and the thickness is T. The first and second microphones M1 and M2 are arranged in the first earphone 31 . The left diagram of FIG. 12B shows a rear view of the first earphone, and the right diagram shows a side view of the first earphone worn on the left ear.

Wherein, the first and second microphones M1 and M2 are generally arranged along the short axis direction of the main body (ellipse or nearly ellipse). When worn on the left ear, there is an acute angle between the short axis and the Z axis (vertical upward direction), and the acute angle may be, for example, 0-30 degrees. In another embodiment, the first and second microphones M1 and M2 in the first earphone 31 can be arranged along the length direction L of the earphone stem, and when worn on the left ear, the length direction is substantially parallel to the Z axis.

As shown in FIG. 12C , according to an embodiment of the present application, the second earphone 32 includes a main body and an earphone stem. The length of the earphone stem is L, the width is W, and the thickness is T. The third and fourth microphones M3 and M4 are arranged in the second earphone 32 . The left diagram of FIG. 12C shows a side view of the second earphone worn on the right ear, and the right diagram shows a rear view of the second earphone.

Wherein, the third and fourth microphones M3 and M4 are generally arranged along the long axis direction of the main body (ellipse or near ellipse). When worn on the right ear, there is an acute angle between the long axis and the X axis (pointing directly behind the wearer's body), and the acute angle can be, for example, 0-30 degrees. In another embodiment, the third and fourth microphones M3 and M4 in the second earphone 32 can be arranged along the width direction W of the earphone stem, and when worn on the right ear, the length direction is substantially parallel to the X axis.

In the case where the first earphone 31 is arranged as shown in FIG. 12B and the second earphone 32 is arranged as shown in FIG. 12C , the arrangement directions of the first and second microphones M1 and M2 are the same as those of the third and fourth microphones M3 and M4. There is an included angle between the arrangement directions, preferably, the included angle is close to 90 degrees.

Since there is an angle between the arrangement directions of the first and second microphones M1, M2 and the arrangement directions of the third and fourth microphones M3, M4, the weighting matrix of the environmental sound processing module can be determined according to the above formula (1), Including determining the weighting matrix of the first environmental sound processing module in the first earphone and the weighting matrix of the second environmental sound processing module in the second earphone, thus while eliminating the omnidirectional background noise, the target direction (wearing directly behind the victim's body) background sound.

As shown in FIG. 12D , according to an embodiment of the present application, a headphone includes a left earphone 33 , a right earphone 34 and a headband 35 connecting and fixing the two earphones. Wherein, the left earphone 33 is provided with first and second microphones, and the right earphone 34 is provided with third and fourth microphones, the first and second microphones are arranged along the first direction, and the third and fourth microphones are arranged along the first direction. Arrangement in two directions. Specifically, the first direction may form a first angle with the central axis of the headset (see FIG. 12D ), and the second direction may form a second angle with the front of the headset. Wherein, the first and second included angles are both acute angles. In one embodiment, after the user wears the headset, the first direction is substantially parallel to the Z-axis, and the second direction is substantially directed directly behind the wearer. Based on the above arrangement, there is an angle close to 90 degrees between the arrangement directions of the first and second microphones and the arrangement directions of the third and fourth microphones.

More specifically, the left earphone includes a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module is composed of the above-mentioned first microphone and the second microphone, and the first noise reduction module can Generate a first noise reduction signal from the audio signal, and perform noise reduction on the received audio signal. In addition, the first ambient sound acquisition module can further acquire the ambient sound signal around the right earphone to enhance the technical effect of eliminating omnidirectional background noise and retaining the background sound directly behind the wearer's body.

It should be understood that although the first earphone is the left earphone and the second earphone is the right earphone, in other embodiments of the application, the first audio receiver (first earphone) may be the right earphone, and the second audio receiver may be the right earphone. The receiver (second earphone) may be the left earphone. In other words, the first and second microphones can be arranged along the X-axis direction, and the third and fourth microphones can be arranged along the Z-axis direction, as long as there is an included angle between the two arrangement directions.

12A-12D described above, the specially arranged microphone array has the function of spatial filtering. If the ambient sound processing module equipped with the microphone array is introduced into the ANC, it can eliminate the omnidirectional background noise while retaining Noise from the target direction can create a new user experience. For example, keeping the ambient sound behind the user while running can improve the safety of the user when wearing the headset. At the same time, under the constraints of signal integrity and signal directivity, the environmental sound processing module optimizes the calculation of beamforming according to the criterion of minimum energy, and finally can obtain the optimal weighting matrix of different frequencies. The matrix can greatly suppress the environmental noise in other directions without losing the sound in the target direction, and under the constraints of the signal directivity index, it can narrow the beam bandwidth of the environmental sound processing module, thereby improving the pertinence of the direction.

It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application. The implementation process constitutes any limitation.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

It should be understood that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may also be distributed to multiple network units . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product may include one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (such as infrared, wireless, microwave, etc.). The computer The readable storage medium can be any available medium that can be read by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated.The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, magnetic tape), optical media (for example, digital video disc (digital video disc, DVD)), or semiconductor media (for example, solid state disk (solid state disk, SSD)), etc.

The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be defined in the claims.

Claims (35)

1. A first earphone, comprising:

   an audio input module configured to receive an audio signal to be played;

   An ambient sound collection module, which includes a first microphone and a second microphone configured to collect ambient sound signals around the first earphone;

   An active noise reduction module configured to generate a noise reduction signal according to the environmental sound signal;

   The environmental sound processing module is configured to obtain the first environmental sound signal of a designated sound pickup direction from the environmental sound signal;

   an audio playing module configured to play the mixed signal of the audio signal, the noise reduction signal and the first ambient sound signal;

   Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the first direction forms a first acute angle with the length direction of the earphone handle of the first earphone.
2. The first earphone according to claim 1, wherein the ambient sound processing module comprises:

   The directional sound pickup module is configured to obtain the sound pickup environmental sound signal of the designated sound pickup direction from the environmental sound signal, and the sound pickup environmental sound signal is a monophonic signal;

   The sound effect processing module is configured to perform sound effect processing on the picked-up environmental sound signal to generate the first environmental sound signal with spatial sound effect.
3. The first earphone according to claim 2, wherein the ambient sound processing module further comprises:

   The sound source direction estimation module is configured to: obtain the direction information of the sound source signal in the specified sound pickup direction;

   The sound effect processing module is configured to: perform sound effect processing on the picked-up environmental sound signal according to the orientation information of the sound source signal to obtain the first environmental sound signal, wherein the first environmental sound signal comes from The spatial sound effect of the orientation of the sound source signal.
4. The first earphone according to claim 3, wherein the sound effect processing module is configured to:

   According to the orientation information of the sound source signal, determine the HRTF filter corresponding to the orientation of the sound source signal;

   Convolving the HRTF filter with the picked-up ambient sound signal to obtain the first ambient sound signal.
5. The first earphone according to claim 1, wherein the ambient sound collection module is further configured to acquire ambient sound signals around a second earphone used in conjunction with the first earphone.
6. The first earphone according to claim 1, wherein the first direction includes a long axis direction of a main body portion of the first earphone.
7. The first earphone according to claim 1, wherein the earphone further comprises:

   The gain adjustment module is configured to adjust the intensity of the first ambient sound signal according to the instruction of the wearer of the earphone.
8. The first earphone according to any one of claims 1-7, wherein the specified sound pickup direction includes a rear of a wearer of the earphone.
9. A second earphone, comprising:

   an audio input module configured to receive an audio signal to be played;

   An ambient sound collection module, which includes a third microphone and a fourth microphone configured to collect ambient sound signals around the second earphone;

   An active noise reduction module configured to generate a noise reduction signal according to the environmental sound signal;

   The environmental sound processing module is configured to obtain the first environmental sound signal of a designated sound pickup direction from the environmental sound signal;

   an audio playing module configured to play the mixed signal of the audio signal, the noise reduction signal and the first ambient sound signal;

   Wherein, the third microphone and the fourth microphone are configured to be arranged along a second direction, and the second direction forms a second acute angle with the width direction of the earphone handle of the second earphone.
10. The second earphone according to claim 9, wherein the ambient sound collection module is further configured to acquire ambient sound signals around the first earphone used with the second earphone.
11. The second earphone according to claim 9, wherein the second direction comprises a minor axis direction of a main body portion of the second earphone.
12. A wireless earphone, comprising the first earphone according to any one of claims 1-8 and the second earphone according to any one of claims 8-11.
13. A wireless earphone, comprising:

    The first audio receiver includes a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module includes a first microphone and a first microphone configured to collect ambient sound signals around the first audio receiver The second microphone, the first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver to reduce noise on the audio signal received by the first audio receiver;

    The second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a third microphone configured to collect ambient sound signals around the second audio receiver A fourth microphone, the second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce noise on the audio signal received by the second audio receiver;

    Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, the first direction forms a first acute angle with the length direction of the handle of the first audio receiver, and the The third microphone and the fourth microphone are configured to be arranged along a second direction forming a second angle belonging to an acute angle with a width direction of the handle of the first audio receiver.
14. The wireless earphone according to claim 13, wherein the first direction is the long axis direction of the main body portion of the first audio receiver, and the second direction is the long axis direction of the main body portion of the second audio receiver. short axis direction.
15. The wireless headset of claim 13, wherein the first direction is orthogonal to the second direction.
16. The wireless earphone according to claim 13, wherein the first ambient sound collection module is further configured to obtain the ambient sound signal around the second audio receiver, and the second ambient sound collection module is also configured to obtain Ambient sound signals around the first audio receiver.
17. The wireless earphone according to claim 13, wherein the first audio receiver further comprises a first environmental sound processing module configured to obtain an environmental sound signal of a designated sound pickup direction from surrounding environmental sound signals, the first environmental sound processing module The second audio receiver also includes a second environmental sound processing module configured to obtain the environmental sound signal of the specified sound pickup direction from the surrounding environmental sound signals.
18. The wireless headset of claim 13, wherein the first audio receiver further includes a first HRTF filter matched to the characteristics of the left ear of the wearer of the wireless headset, and the second audio receiver further includes A second HRTF filter matched to right ear characteristics of a wearer of the wireless earphone is included.
19. The wireless headset according to any one of claims 13-18, wherein the specified sound pickup direction includes the rear of the wearer of the wireless headset.
20. A headset, comprising:

    The first audio receiver includes a first ambient sound collection module and a first noise reduction module, wherein the first ambient sound collection module includes a first microphone and a first microphone configured to collect ambient sound signals around the first audio receiver The second microphone, the first noise reduction module is configured to generate a first noise reduction signal according to the ambient sound signal around the first audio receiver to reduce noise on the audio signal received by the first audio receiver;

    The second audio receiver includes a second ambient sound collection module and a second noise reduction module, wherein the second ambient sound collection module includes a third microphone and a third microphone configured to collect ambient sound signals around the second audio receiver A fourth microphone, the second noise reduction module is configured to generate a second noise reduction signal according to the ambient sound signal around the second audio receiver to reduce noise on the audio signal received by the second audio receiver;

    Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the third microphone and the fourth microphone are configured to be arranged along a second direction, wherein the first direction and the head The central axis of the headset forms a first angle that is an acute angle, and the second direction forms a second angle that is an acute angle with the front of the headset.
21. The headset of claim 20, wherein the first direction is orthogonal to the second direction.
22. The headset according to claim 20, wherein the first ambient sound collection module is further configured to acquire ambient sound signals around the second audio receiver, and the second ambient sound collection module is also configured to Acquire ambient sound signals around the first audio receiver.
23. The headset according to claim 20, wherein the first audio receiver further comprises a first environmental sound processing module configured to acquire an environmental sound signal of a designated sound pickup direction from surrounding environmental sound signals, The second audio receiver further includes a second environmental sound processing module configured to obtain the environmental sound signal of the specified sound pickup direction from the surrounding environmental sound signals.
24. A user equipment, comprising the wireless headset according to any one of claims 13-20 or the headset according to any one of claims 21-23.
25. An audio system, comprising a sound source, and the wireless earphone according to any one of claims 13-19 or the headset according to any one of claims 20-23.
26. A method of processing an audio signal, wherein the method is applied to a first earphone, the method comprising the steps of:

    S1. Receive an input audio signal;

    S2. Using the first microphone and the second microphone in the first earphone to collect the ambient sound signal around the first earphone; wherein, the first microphone and the second microphone are configured to be arranged along a first direction, The first direction is substantially parallel to the direction directly behind the wearer or substantially parallel to the vertical direction of the first earphone;

    S3. Generate a first noise reduction signal based on the environmental sound signal;

    S4. Obtaining a first environmental sound signal in a specified pickup direction from the environmental sound signal; and

    S5. Outputting an audio signal to be played based on the input audio signal, the first noise reduction signal, and the first ambient sound signal.
27. The method according to claim 26, wherein said step S4 comprises:

    Acquiring a sound-picking environmental sound signal of the designated sound-picking direction from the environmental sound signal, wherein the sound-picking environmental sound signal is a monaural signal;

    Sound effect processing is performed on the picked-up environmental sound signal to generate the first environmental sound signal with spatial sound effect.
28. The method according to claim 27, further comprising:

    Obtaining the orientation information of the sound source signal in the specified pickup direction;

    Perform sound effect processing on the picked-up environmental sound signal according to the orientation information of the sound source signal to obtain the first environmental sound signal, wherein the first environmental sound signal has an orientation from the sound source signal Spatial sound.
29. The method according to claim 28, further comprising:

    According to the orientation information of the sound source signal, determine the HRTF filter corresponding to the orientation of the sound source signal;

    Convolving the HRTF filter with the picked-up ambient sound signal to obtain the first ambient sound signal.
30. The method according to claim 29, further comprising:

    acquiring an ear image of a wearer of the headset;

    extracting ear features from the ear image;

    The HRTF filter is selected to match the ear characteristics.
31. The method according to claim 26, further comprising:

    Under the premise that the signal integrity index of the environmental sound processing module is equal to the number of microphones in the first earphone and the signal directivity index of the environmental sound processing module is equal to the square of the number of microphones, based on making the first environmental sound signal The weighting matrix of the environmental sound processing module is determined by minimizing the energy, wherein the environmental sound processing module is configured to obtain the first environmental sound signal from the environmental sound signal.
32. The method according to any one of claims 26-31, further comprising:

    The strength of the first ambient sound signal is adjusted according to the instruction of the wearer of the earphone.
33. A method for processing an audio signal, applied to a wireless stereo headset, wherein the method includes:

    S1. Using the first earphone and the second earphone to receive the first audio signal and the second audio signal respectively;

    S2. Utilize the first microphone and the second microphone in the first earphone to collect the ambient sound signal around the first earphone, and use the third microphone and the fourth microphone in the second earphone to collect the second Ambient sound signal around the earphone;

    Wherein, the first microphone and the second microphone are configured to be arranged along a first direction, and the third microphone and the fourth microphone are configured to be arranged along a second direction, wherein the first direction is substantially parallel in the direction directly behind the wearer, and the second direction is substantially parallel to the vertical direction of the second earphone;

    S3. Generate a first noise reduction signal and a second noise reduction signal based on the environmental sound signal around the first earphone and the environmental sound signal around the second earphone;

    S4. Obtain the first environmental sound signal specifying the sound pickup direction from the environmental sound signal around the first earphone, and obtain the second environmental sound signal specifying the sound pickup direction from the environmental sound signal around the second earphone ;as well as

    S5. Generate a mixed signal of the first audio signal, the first noise reduction signal and the first ambient sound signal, and generate the second audio signal, the second noise reduction signal and the first noise reduction signal Mixing signal of the second ambient sound signal.
34. The method of claim 33, further comprising:

    Obtaining the orientation information of the sound source signal in the specified pickup direction;

    determining a first angular change of the sound source signal relative to a direction directly behind the wearer and a second angular change relative to a vertical direction of the second earphone;

    Determine an attenuation matrix of a microphone array based on the first angle change and the second angle change, wherein the microphone array includes the first microphone, the second microphone, the third microphone, and the first microphone Four microphones.
35. The method of claim 34, further comprising:

    Under the premise that the signal integrity index of the environmental sound processing module is equal to the number of microphones in the microphone array and the signal directivity index of the environmental sound processing module is equal to the square of the number of microphones, based on making the first environmental sound The energy of the signal is the smallest and the energy of the second environmental sound signal is the smallest to determine the weighting matrix of the environmental sound processing module, wherein the environmental sound processing module is configured to obtain the second environmental sound signal from the environmental sound signal An ambient sound signal and/or the second ambient sound signal.

Images