CN116709154B

CN116709154B - Sound field calibration method and related device

Info

Publication number: CN116709154B
Application number: CN202211311947.5A
Authority: CN
Inventors: 晏细猫; 胡贝贝; 许剑峰
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2022-10-25
Filing date: 2022-10-25
Publication date: 2024-04-09
Anticipated expiration: 2042-10-25
Also published as: CN116709154A

Abstract

The embodiment of the application provides a sound field calibration method and a related device, and relates to the technical field of terminals. The method comprises the following steps: the terminal equipment acquires information between a user and the terminal equipment, wherein the information comprises: the method comprises the steps of obtaining distance information between terminal equipment and a user, an included angle between the terminal equipment and the user in the horizontal direction and an included angle between the terminal equipment and the user in the vertical direction; according to the information, the terminal equipment carries out voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal equipment to obtain a second audio signal; the terminal device plays the second audio signal, wherein the distance between the sound field position of the second audio signal and the center position of the terminal device is smaller than the first preset value. Therefore, the sound field position is close to the center position of the terminal equipment, so that the influence of sound field change caused by the position and distance change of the user is reduced, and the sound effect experience of the user is improved.

Description

Sound field calibration method and related device

Technical Field

The application relates to the technical field of terminals, in particular to a sound field calibration method and a related device.

Background

With the development of terminal technology, users can play video and audio through various terminal devices. When a user watches video and audio, the user can not always watch the video and audio at the same position, and the user can adjust the watching position at any time.

However, after the user adjusts the viewing position, the sound effect obtained by the user may be deteriorated due to the change of the position, affecting the user experience.

Disclosure of Invention

The embodiment of the application provides a sound field calibration method and a related device, wherein an audio algorithm module of terminal equipment can adjust the position of a sound field through human voice adjustment, horizontal sound field calibration and vertical sound field calibration on an input audio signal, so that the position of the sound field is close to the center position of the terminal equipment, the influence of sound field change caused by user position and distance change is reduced, and the sound effect experience of a user is improved.

In a first aspect, an embodiment of the present application provides a method for sound field calibration, the method including:

the terminal equipment acquires information between a user and the terminal equipment, wherein the information comprises the following components: the method comprises the steps of obtaining distance information between terminal equipment and a user, an included angle between the terminal equipment and the user in the horizontal direction and an included angle between the terminal equipment and the user in the vertical direction; according to the information, the terminal equipment carries out voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal equipment to obtain a second audio signal; the terminal device plays the second audio signal, wherein the distance between the sound field position of the second audio signal and the center position of the terminal device is smaller than the first preset value. Therefore, the sound field position is close to the center position of the terminal equipment, so that the influence of sound field change caused by the position and distance change of the user is reduced, and the sound effect experience of the user is improved.

In a possible implementation manner, according to information, the terminal device performs voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal device to obtain a second audio signal, which may include: the terminal equipment performs voice extraction on the first audio signal to obtain a voice signal and a background sound signal; the terminal equipment adjusts the voice of the voice signal according to the distance information to obtain a compensated voice signal; the terminal equipment mixes the background sound signal and the compensated voice signal to obtain a mixed sound signal; the terminal equipment carries out horizontal offset calibration of the sound field on the mixed sound signal according to the included angle of the horizontal direction; and the terminal equipment carries out vertical offset calibration of the sound field on the mixed sound signal according to the vertical included angle. Thus, the audio signal processed by the audio algorithm module can enable the sound field position to be near the center position of the terminal equipment, so that a user can have better sound field experience.

In a possible implementation manner, the terminal device performs voice adjustment on the voice signal according to the distance information to obtain a compensated voice signal, which may include: the terminal equipment determines an attenuation coefficient according to the distance information; when the distance information is larger than or equal to a second preset value, the terminal equipment performs gain compensation on the voice signal according to the attenuation coefficient; and when the distance information is smaller than a second preset value, the terminal equipment carries out gain attenuation on the voice signal according to the attenuation coefficient. Therefore, the attenuation coefficient is adjusted according to different distance information, and the gain of the input voice signal can be further adjusted, so that the adjusted audio signal output by the terminal equipment is more in line with the experience of the user when the position of the user changes.

In a possible implementation manner, the terminal device performs horizontal offset calibration of the sound field on the mixed signal according to the horizontal direction included angle, and may include: the terminal equipment determines a directivity function in the horizontal direction according to the included angle in the horizontal direction; and the terminal equipment carries out horizontal offset calibration of the sound field on the mixed sound signal according to the directivity function in the horizontal direction. Therefore, the terminal equipment correspondingly adjusts the directivity of the sound field based on the horizontal included angle between the terminal equipment and the user, so that the calibration of the sound field in the horizontal direction is realized, and the user has better sound field experience in the scenes such as film watching.

In a possible implementation manner, the terminal device may include a first speaker and a second speaker. The first speaker and the second speaker are located at the bottom of the terminal device, and the first speaker and the second speaker are maintained on the same horizontal line. Thus, the terminal equipment can have better sound effect.

In a possible implementation manner, the terminal device performs vertical offset calibration of the sound field on the mixed signal according to the vertical included angle, and may include: the terminal equipment determines a directivity function in the vertical direction according to the included angle in the vertical direction; and the terminal equipment performs vertical offset calibration of the sound field on the mixed signal according to the directivity function in the vertical direction. Therefore, the terminal equipment correspondingly adjusts the directivity of the sound field based on the included angle between the terminal equipment and the user in the vertical direction, so that the calibration of the sound field in the vertical direction is realized, and the user has better sound field experience in the scenes such as film watching.

In a possible implementation manner, the terminal device may include a third speaker and a fourth speaker. The third speaker and the fourth speaker are located at the top of the terminal device, and the third speaker and the fourth speaker are maintained on the same horizontal line. Thus, the terminal equipment can have better sound effect.

In a possible implementation manner, the terminal device obtains information between the user and the terminal device, which may include: the terminal equipment obtains information between the user and the terminal equipment based on the camera. Therefore, the real-time calibration of the sound field can be realized according to the position information of the user and the linkage of the audio algorithm module, so that the position of the sound field is near the center position of the terminal equipment, and the user experience is improved.

In a second aspect, an embodiment of the present application provides a sound field calibration apparatus, where a terminal device may include: the processing unit is used for acquiring information between the user and the terminal equipment, wherein the information comprises: the method comprises the steps of obtaining distance information between terminal equipment and a user, an included angle between the terminal equipment and the user in the horizontal direction and an included angle between the terminal equipment and the user in the vertical direction; according to the information, the processing unit is further used for performing voice adjustment, horizontal sound field calibration and vertical sound field calibration on the first audio signal to be played by the terminal equipment to obtain a second audio signal; and the processing unit plays the second audio signal, wherein the distance between the sound field position of the second audio signal and the center position of the terminal equipment is smaller than a first preset value.

In a possible implementation manner, the processing unit is configured to perform voice extraction on the first audio signal to obtain a voice signal and a background sound signal; the voice adjusting device is also used for adjusting the voice of the voice signal according to the distance information to obtain a compensated voice signal; the method is also used for mixing the background sound signal and the compensated human sound signal to obtain a mixed sound signal; the method is particularly used for carrying out horizontal offset calibration of the sound field on the sound mixing signal according to the included angle of the horizontal direction; the method is particularly used for calibrating the vertical offset of the sound field of the sound mixing signal according to the included angle of the vertical direction.

In a possible implementation, the processing unit is configured to determine the attenuation coefficient according to the distance information; when the distance information is greater than or equal to a second preset value, the processing unit is further used for performing gain compensation on the voice signal according to the attenuation coefficient. When the distance information is smaller than a second preset value, the processing unit is further used for carrying out gain attenuation on the voice signal according to the attenuation coefficient.

In a possible implementation manner, the processing unit is configured to determine a directivity function of the horizontal direction according to the included angle of the horizontal direction; and the processing unit is also used for carrying out horizontal offset calibration of the sound field on the mixed sound signal according to the directivity function in the horizontal direction.

In one possible implementation, the terminal device includes a first speaker, a second speaker, wherein the first speaker and the second speaker are located at a bottom of the terminal device, and the first speaker and the second speaker are maintained on the same horizontal line.

In a possible implementation manner, the processing unit is configured to determine a directivity function of the vertical direction according to the included angle of the vertical direction; and the processing unit is also used for carrying out vertical offset calibration on the sound field on the mixed sound signal according to the directivity function in the vertical direction.

In one possible implementation, the terminal device includes a third speaker, a fourth speaker, wherein the third speaker and the fourth speaker are located on top of the terminal device, and the third speaker and the fourth speaker remain on the same horizontal line.

In a possible implementation manner, the processing unit is further configured to obtain information between the user and the terminal device based on the camera.

In a third aspect, embodiments of the present application provide a terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, when executing the computer program, causing the terminal device to perform the sound field calibration method as described in the first aspect or any implementation manner of the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing instructions that, when executed, cause a computer to perform a sound field calibration method as described in the first aspect or any implementation of the first aspect.

In a fifth aspect, a computer program product comprising a computer program which, when run, causes a computer to perform the sound field calibration method as described in the first aspect or any implementation of the first aspect.

It should be understood that, the second aspect to the fifth aspect of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects obtained by each aspect and the corresponding possible embodiments are similar, and are not repeated.

Drawings

Fig. 1 is a schematic structural diagram of a terminal device provided in an embodiment of the present application;

fig. 2 is a schematic frame diagram of a sound field calibration method according to an embodiment of the present application;

fig. 3 is a schematic diagram of sound field change when a user approaches a terminal device according to an embodiment of the present application;

fig. 4 is a schematic diagram of sound field change when a user is far away from a terminal device according to an embodiment of the present application;

fig. 5 is a schematic diagram of sound field change when a user is located at a side of a terminal device according to an embodiment of the present application;

Fig. 6 is a schematic diagram of sound field change when a user is located above a terminal device according to an embodiment of the present application;

fig. 7 is a schematic diagram of sound field change when a user is located below a terminal device according to an embodiment of the present application;

fig. 8 is a schematic diagram of a specific flow of processing an audio signal by an audio algorithm module according to an embodiment of the present application;

fig. 9 is a schematic diagram of a sound field calibration method according to an embodiment of the present application;

fig. 10 is a schematic diagram of gain adjustment of a signal by a filter according to an embodiment of the present application;

fig. 11 is a schematic view of a scenario in which a sound source generates sound pressure to a certain point according to an embodiment of the present application;

fig. 12 is a schematic diagram of placement of a speaker array in a terminal device according to an embodiment of the present application;

fig. 13 is a schematic diagram of beam forming simulation of a point sound source with different frequencies according to an embodiment of the present application;

fig. 14 is a schematic diagram of simulation of beam deflection according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of a sound field calibration device according to an embodiment of the present application;

fig. 16 is a schematic hardware structure of another terminal device according to an embodiment of the present application.

Detailed Description

In order to facilitate the clear description of the technical solutions of the embodiments of the present application, the following will simply introduce some terms related to the embodiments of the present application:

1. Terminology

In the embodiments of the present application, the words "first," "second," and the like are used to distinguish between identical or similar items that have substantially the same function and effect. For example, the first chip and the second chip are merely for distinguishing different chips, and the order of the different chips is not limited. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a alone, a and B together, and B alone, wherein a, B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

2. Terminal equipment

The terminal device may also be referred to as: a terminal (terminal), a User Equipment (UE), a Mobile Station (MS), a Mobile Terminal (MT), and the like. The terminal device may be a large screen product such as a smart tv, a tablet (Pad), a notebook, or a smart screen product having at least two play devices, or the terminal device may be a mobile phone (mobile phone), a wearable device, a Virtual Reality (VR) terminal device, an augmented reality (augmented reality, AR) terminal device, a wireless terminal in industrial control (industrial control), a wireless terminal in unmanned (self-driving), a wireless terminal in remote operation (remote medical surgery), a wireless terminal in smart grid (smart grid), a wireless terminal in transportation security (transportation safety), a wireless terminal in smart city (smart city), a wireless terminal in smart home (smart home), or the like.

Fig. 1 illustrates a schematic structural diagram of a terminal device according to an embodiment of the present application.

The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, an indicator 191, a camera 192, a display 193, and the like.

It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, such as: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, the instruction or data can be directly called from the memory, repeated access is not needed, and the waiting time of the processor 110 is reduced, so that the efficiency of the system is improved.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to realize expansion of the memory capability of the terminal device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 121 may be used to store computer-executable program code that includes instructions. The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an application program (such as a sound playing function, an image playing function, etc.) required for at least one function of the operating system, etc. The storage data area may store data created during use of the terminal device (such as audio data, phonebook, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (universal flash storage, UFS), and the like. The processor 110 performs various functional applications of the terminal device and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor. For example, the sound field calibration method of the embodiments of the present application may be performed.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge a terminal device, or may be used to transfer data between the terminal device and a peripheral device. And can also be used for connecting with a headset, and playing audio through the headset. The interface may also be used to connect other terminal devices, such as AR devices, etc.

The charge management module 140 is configured to receive a charge input from a charger. The charger can be a wireless charger or a wired charger. The power management module 141 may be used to connect the charge management module 140 with the processor 110.

The wireless communication function of the terminal device may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Antennas in the terminal device may be used to cover single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 may provide a solution for wireless communication including 2G/3G/4G/5G or the like applied on a terminal device. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA), etc. The mobile communication module 150 may receive electromagnetic waves from the antenna 1, perform processes such as filtering, amplifying, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation.

The wireless communication module 160 may provide solutions for wireless communication including wireless local area network (wirelesslocal area networks, WLAN) (e.g., wireless fidelity (wireless fidelity, wi-Fi) network), bluetooth (BT), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), etc. as applied on a terminal device.

The terminal device may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as audio playback or recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The speaker 170A, also called "horn", is used to convert an audio electrical signal into a sound signal, and 1 or N speakers 170A, N being a positive integer greater than 1, may be included in the terminal device. The terminal device can listen to music, video, or hands-free conversation through the speaker 170A, etc. A receiver 170B, also referred to as a "earpiece", is used to convert the audio electrical signal into a sound signal. When the terminal device picks up a call or voice message, the voice can be picked up by placing the receiver 170B close to the human ear.

In this embodiment of the present application, the terminal device may set a plurality of playing devices, where the playing devices may include: speaker 170A and/or receiver 170B. In a scenario where the terminal device plays video, at least one speaker 170A and/or at least one receiver 170B simultaneously plays audio signals.

The earphone interface 170D is used to connect a wired earphone. Microphone 170C, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. In this embodiment of the present application, the terminal device may receive the sound signal for waking up the terminal device based on the microphone 170C, and convert the sound signal into an electrical signal that may be processed later, such as voiceprint data described in the embodiment of the present application, and the terminal device may have at least one microphone 170C.

The sensor module 180 may include a distance sensor, a temperature sensor, or an ambient light sensor, etc. (not shown in fig. 3).

The keys 190 include a power-on key, a volume key, etc. The keys 190 may be mechanical keys or touch keys. The terminal device may receive key inputs, generating key signal inputs related to user settings of the terminal device and function control. The indicator 191 may be an indicator light, may be used to indicate a state of charge, or a change in power, etc., and may be used to indicate a message, missed call, notification, etc.

The camera 192 is used to capture still images or video. In some embodiments, the terminal device may include 1 or N cameras 192, N being a positive integer greater than 1.

The display 193 is used to display images, videos, and the like. The display 193 includes a display panel. In some embodiments, the terminal device may include 1 or N displays 193, N being a positive integer greater than 1.

The terminal device implements display functions through a GPU, a display screen 193, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 193 and an application processor.

The software system of the terminal device may adopt a layered architecture, an event driven architecture, a microkernel architecture, a microservice architecture, a cloud architecture, or the like, which will not be described herein.

With the development of terminal technology, users can play video and audio through various terminal devices. When a user views the video, the user can adjust the position of viewing the video at any time, for example, the user can adjust the distance from the terminal device, the angle in the horizontal direction and the angle in the vertical direction. The change of the distance or angle between the user and the terminal equipment can influence the change of the sound field of the video and audio, so that the problem that the sound field is not centered can be brought, and the user cannot obtain the sound effect experience of the centering of the sound field.

In one implementation, the terminal device may balance the left and right channels by adjusting the volume of the left and right channels. However, the problem that the sound field output by the terminal equipment is not centered still exists, and the user experience is affected.

In view of this, in the sound field calibration method provided in the embodiment of the present application, as shown in fig. 2, a camera of a terminal device or a device of other terminal devices may perform positioning and motion locking on a user, obtain distance information, a horizontal direction included angle and a vertical direction included angle between the user and the terminal device, and transmit the distance information, the horizontal direction included angle and the vertical direction included angle to an audio algorithm module of the terminal device. The audio algorithm module can adjust the position of the sound field by carrying out human voice adjustment, horizontal sound field calibration and vertical sound field calibration on the input audio signal, so that the position of the sound field is close to the center position of the terminal equipment, the influence of sound field change caused by user position and distance change is reduced, and the sound effect experience of a user is improved.

Illustratively, the user may adjust the viewing position while the user views the audio-visual. The user-adjusted position relative to the terminal device may have the following scenarios:

Scene one: the distance between the terminal device and the user varies.

The change in distance between the terminal device and the user may include the user being proximate to the terminal device.

As shown in fig. 3, when the user is in the position in a of fig. 3, the sound field of the terminal device may be located in the center of the terminal device, and at this time, the user may have a better sound field experience when watching the audio and video. When the user approaches the terminal device, as shown in b of fig. 3, the distance between the user and the terminal device decreases, so that the distance between the user and the sound field also decreases, and the user may feel that the sound field is closer. At this time, the terminal device may perform gain adjustment on the sound field through the audio algorithm module, as shown in c of fig. 3, so that the user may feel that the adjusted sound field is located near the center position of the terminal device. It can be understood that the gain adjustment of the sound field by the terminal device may also be referred to as performing a remote processing on the sound field, so that the sound field is properly pulled out, thereby improving the user experience.

The change in distance between the terminal device and the user may also include the user being remote from the terminal device.

As shown in fig. 4, when the user is in the position in a of fig. 4, the sound field of the terminal device may be located in the center of the terminal device, and at this time, the user may have a better sound field experience when watching the audio and video. When the user is away from the terminal device, as shown in b of fig. 4, the distance between the user and the terminal device increases, so that the distance between the user and the sound field increases, and the user may feel the remote sound field. At this time, the terminal device may perform gain adjustment on the sound field through the audio algorithm module, as shown in c of fig. 4, so that the user may feel that the adjusted sound field is located near the center position of the terminal device. It can be understood that the gain adjustment of the sound field by the terminal device may also be referred to as zooming in on the sound field, so that the sound field is properly close to each other, thereby improving the user experience.

Scene II: the angle in the horizontal direction between the terminal device and the user varies.

The change in angle in the horizontal direction between the terminal device and the user may comprise the user being located to the left or to the right of the terminal device.

As shown in fig. 5, when the user is in the position in a of fig. 5, the sound field of the terminal device may be located in the center of the terminal device, and at this time, the user may have a better sound field experience when watching the audio and video. For example, when the user is not facing the terminal device, but is located on the left side of the terminal device as shown in b of fig. 5, an angle in the horizontal direction may be generated between the user and the terminal device, so that the sound field is biased toward the direction in which the user is located. At this time, the terminal device may perform horizontal offset calibration on the sound field through the audio algorithm module, as shown in c of fig. 5, so that the user may feel that the adjusted sound field is located near the center position of the terminal device.

Scene III: the angle in the vertical direction between the terminal device and the user varies.

The change in angle in the vertical direction between the terminal device and the user may comprise the height of the user's ear above a speaker in the terminal device.

As shown in fig. 6, when the user is in the position in a of fig. 6, the height of the ear of the user may be kept flush with the height of the speaker, and the sound field of the terminal device may be located in the center of the terminal device, at this time, the user may have a better sound field experience when watching audio and video. When the user's ear is higher than the speaker, as shown in fig. 6 b, a vertical angle may be created between the user and the terminal device, biasing the sound field below the terminal device. At this time, the terminal device may perform vertical offset calibration on the sound field through the audio algorithm module, as shown in fig. 6 c, so that the position of the sound field may be moved upward, and thus the user may feel that the adjusted sound field is located near the center position of the terminal device.

The change in angle in the vertical direction between the terminal device and the user may comprise the height of the user's ear being located below a speaker in the terminal device.

As shown in fig. 7, when the user is in the position in a of fig. 7, the height of the ear of the user may be kept flush with the height of the speaker, and the sound field of the terminal device may be located in the center of the terminal device, at this time, the user may have a better sound field experience when watching audio and video. When the user's ear is at a lower level than the speaker, as shown in fig. 7 b, a vertical angle may be created between the user and the terminal device, biasing the sound field above the terminal device. At this time, the terminal device may perform vertical offset calibration on the sound field through the audio algorithm module, as shown in fig. 7 c, so that the position of the sound field may be moved downward, and thus the user may feel that the adjusted sound field is located near the center position of the terminal device.

In summary, when the position of the user relative to the terminal device changes, the terminal device may process the audio signal through the audio algorithm module. The processing of the audio signal by the audio algorithm module may include: gain adjustment of the audio signal, horizontal offset calibration of the sound field, and vertical offset calibration of the sound field.

Specifically, as shown in fig. 8, a specific flow of processing an audio signal by the audio algorithm module may include:

the terminal device inputs an audio signal to the voice extraction module, and the voice extraction module can separate the voice signal and the background sound signal in the input audio signal after receiving the input audio signal. Meanwhile, the voice extraction module can transmit the obtained voice signal to the middle-high frequency compensation module, and the middle-high frequency compensation module can compensate or attenuate the gain of the voice signal. The human voice extraction module can also transmit background sound signals to the sound field expansion module, and the sound field expansion module can widen the width of a sound field. It will be appreciated that if there is no human voice signal in the input audio signal, the mid-to-high frequency compensation module will not perform the gain compensation or attenuation process.

In a possible implementation, the voice extraction module may implement voice signal extraction based on a voice extraction model of a neural network, for example, the voice extraction module may implement voice signal extraction through a full convolution time domain audio separation network Conv-Tasnet, or a frequency domain audio separation network Demucs, etc. The voice extraction module can also realize voice signal extraction based on other algorithms, and the voice signal extraction mode is not limited in the embodiment of the application.

In the embodiment of the application, the middle-high frequency compensation module can perform middle-high frequency compensation or attenuation in a certain proportion according to the distance information between the camera and the target user provided by the camera, so that the sound field distance is calibrated. The sound field expansion module can expand the sound field width based on the speaker layout, and can also adopt sound string elimination, virtual speaker creation or other methods to expand the sound field width.

After the middle-high frequency compensation module completes gain compensation or attenuation processing on the voice signal, the middle-high frequency compensation module can output the voice signal to the sound mixing module. After the sound field expansion module processes the background sound signal, the sound field expansion module can also output the background sound signal to the sound mixing module. After receiving the processed human voice signal and the background sound signal with the expanded sound field, the sound mixing module can mix the human voice signal and the background sound signal back into stereo.

Further, the mixing module may transmit the stereo signal to the base sound effect module. The base sound effect module may include: a pre-gain module, an equalization module, a linear phase calibration module, a multi-sub dynamic range control (multiband dynamic range control, MBDRC) module, and/or a dynamic range control (dynamic range control, DRC) module, etc.

The front gain module may include an Equalizer (EQ), and its main function may include to control the gain level of the input audio signal. The equalization module may be used to control the timbre of the input audio signal. A Linear Phase (LPH) calibration module may compensate for some of the drawbacks of the speakers, e.g., if two speakers are included in the terminal device, a linear phase calibration module may compensate for the consistency calibration problem of the two speakers, etc. The MBDRC module may match different gain amounts according to each audio segment, thereby improving the volume effect of the output audio signal. Illustratively, when there are some noise in the audio signal, the MBDRC module may segment to control the volume of the audio signal, thereby controlling the noise in the audio signal. The DRC module may maintain the audio signal at a maximum level, increase the volume of the small signal in the audio signal, and thereby increase the loudness of the audio signal.

After each module in the basic sound effect module processes the audio signal, the basic sound effect module can transmit the processed audio signal to the sound field horizontal offset calibration module and the sound field vertical offset calibration module. The core module in the sound field horizontal offset calibration module uses a beam forming algorithm, and the sound field horizontal offset calibration module can calibrate the audio information in the horizontal direction based on the beam forming algorithm according to the user horizontal offset angle provided by a camera or other devices in the terminal equipment, so that the horizontal calibration of the sound field is realized. The core module in the sound field vertical deviation calibration module also uses a beam forming algorithm, and the sound field vertical deviation calibration module can calibrate the audio information in the vertical direction based on the beam forming algorithm according to the user vertical deviation angle provided by a camera or other devices in the terminal equipment, so that the vertical calibration of the sound field is realized. In this way, the sound field can be located near the center position of the terminal device by the sound field horizontal offset calibration module and the sound field vertical offset calibration module.

After the sound field horizontal offset calibration module and the sound field vertical offset calibration module finish sound field calibration on the audio signal, the calibrated audio signal can be transmitted to a Limiter (LMT), and the limiter can limit the signal amplitude of the output audio within a certain range. After the audio signal passes through the limiter, it may be transmitted to a post gain module, which may be used to control the gain level of the audio signal. After the post gain module processes the audio information, the terminal device can obtain the output audio signal.

The sound field calibration method according to the embodiment of the present application is described in detail below by way of specific examples. The following embodiments may be combined with each other or implemented independently, and the same or similar concepts or processes may not be described in detail in some embodiments.

Fig. 9 illustrates a sound field calibration method according to an embodiment of the present application. The method comprises the following steps:

s901, the terminal equipment acquires information between a user and the terminal equipment, wherein the information can comprise: distance information between the terminal equipment and the user, a horizontal direction included angle between the terminal equipment and the user, and a vertical direction included angle between the terminal equipment and the user.

In the embodiment of the application, the terminal equipment can acquire the information between the user and the terminal equipment based on the camera or other sensors. The number of cameras in the terminal equipment can be 1 or N, and N is a positive integer greater than 1. Other sensors may include microphone devices, bluetooth devices, or ultrasonic devices, etc., and specific sensor types are not limited in this application.

Taking the example that the camera obtains the information between the user and the terminal device, in one possible implementation manner, the camera can obtain the information of the user based on a face recognition algorithm. For example, the camera may feed back a coordinate point where the face of the user is located in the image to the terminal device, and the terminal device may calculate the position information, the horizontal direction included angle, and the vertical direction included angle of the user through coordinate conversion based on the coordinate point. In another possible implementation manner, when two cameras are provided in the terminal device, the two cameras can synthesize the photographed images into a three-dimensional stereoscopic image, so as to extract coordinates of the face outline of the user. The terminal equipment can calculate the distance information and the angle information between the user and the terminal equipment through a binocular camera calculation principle. The terminal device may also obtain the information between the user and the terminal device in other manners, and the manner of obtaining the user location information in the embodiment of the present application is not limited.

It may be understood that the distance information between the terminal device and the user may be the distance between the camera and the face of the user, or the distance between the camera and other parts of the body of the user, which is not limited in the embodiment of the present application. The horizontal included angle between the terminal equipment and the user can be the change of the horizontal included angle between the camera and the face of the user or the change of the horizontal included angle between the camera and other parts of the body of the user. The embodiments of the present application are not limited. The vertical included angle between the terminal equipment and the user can be the change of the vertical included angle between the camera and the face of the user, or the change of the vertical included angle between the camera and other parts of the body of the user. The embodiments of the present application are not limited.

S902, according to the information, the terminal equipment adjusts the voice of the first audio signal to be played by the terminal equipment, corrects the horizontal sound field and corrects the vertical sound field, and obtains a second audio signal.

In this embodiment of the present application, the first audio signal may be an audio signal that is not processed by the audio algorithm module in the terminal device. The first audio signal may be, for example, the input audio in the embodiment corresponding to fig. 8. The second audio signal may be an audio signal processed by an audio algorithm module in the terminal device, and the second audio signal may be output audio in the embodiment corresponding to fig. 8.

The terminal equipment can adjust the human voice through a middle-high frequency compensation module in the audio algorithm module, perform horizontal sound field calibration on the first audio signal through a sound field horizontal offset calibration module, and perform vertical sound field calibration on the first audio signal through a sound field vertical offset calibration module. Specifically, the process of obtaining the second audio signal from the first audio signal by the terminal device may refer to the specific flow description of processing the audio signal by the audio algorithm module in the embodiment corresponding to fig. 8, which is not repeated.

S903, the terminal equipment plays a second audio signal, wherein the distance between the sound field position of the second audio signal and the center position of the terminal equipment is smaller than a first preset value.

In this embodiment of the present application, the first preset value may be a difference between a sound field position preset in the terminal device and a center position of the terminal device. The first preset value may be the same or different in different terminal devices, and may be set according to an empirical preset value obtained by analyzing data. The specific value of the first preset value is not limited herein. It will be appreciated that the first preset value may be a small value, and in an ideal case, the sound field may be located at the same position as the center of the terminal device, where the sound effect of the sound field received by the user is the best.

When the user is in different positions, the terminal equipment can enable the sound field position to be near the center position of the terminal equipment through human voice adjustment, horizontal offset calibration of the sound field and vertical offset calibration of the sound field to the audio signal, so that the user can have better sound field experience.

Optionally, based on the embodiment corresponding to fig. 9, the step of performing, by the terminal device of S902, voice adjustment, horizontal sound field calibration, and vertical sound field calibration on the first audio signal to be played by the terminal device according to the information to obtain a second audio signal may include: the terminal equipment performs voice extraction on the first audio signal to obtain a voice signal and a background sound signal; the terminal equipment adjusts the voice of the voice signal according to the distance information to obtain a compensated voice signal; the terminal equipment mixes the background sound signal and the compensated voice signal to obtain a mixed sound signal; the terminal equipment carries out horizontal offset calibration of the sound field on the mixed sound signal according to the included angle of the horizontal direction; and the terminal equipment carries out vertical offset calibration of the sound field on the mixed sound signal according to the vertical included angle.

In this embodiment of the present application, the process of processing the first audio signal by the specific terminal device to obtain the second audio signal may refer to the specific flow description of processing the audio signal by the audio algorithm module in the embodiment corresponding to fig. 8, which is not repeated. The audio signal processed by the audio algorithm module can enable the sound field position to be near the center position of the terminal equipment, so that a user can have better sound field experience.

Optionally, on the basis of the embodiment corresponding to fig. 9, the terminal device performs voice adjustment on the voice signal according to the distance information to obtain the compensated voice signal, which may include: the terminal equipment determines an attenuation coefficient according to the distance information; when the distance information is larger than or equal to a second preset value, the terminal equipment performs gain compensation on the voice signal according to the attenuation coefficient; and when the distance information is smaller than a second preset value, the terminal equipment carries out gain attenuation on the voice signal according to the attenuation coefficient.

In this embodiment of the present application, the second preset value may be a distance between the terminal device and the user, and under the distance, the position of the sound field better accords with the experience of the user. It can be understood that the second preset value may be a preset value in the terminal device, and the second preset values corresponding to different terminal devices may be the same or different. The second preset value may be set according to an empirical preset value obtained by analyzing the data. The specific magnitude of the second preset value is not limited in the embodiments of the present application.

Taking the second preset value as 2 meters as an example, when the distance between the terminal equipment and the user is greater than or equal to 2 meters, the attenuation coefficient A obtained by the terminal equipment _a If so, the terminal device needs to perform gain compensation on the voice signal. When the distance between the terminal equipment and the user is smaller than 2 meters, the attenuation coefficient A obtained by the terminal equipment _a And negative, at this time, the terminal device needs to perform gain attenuation on the voice signal. The gain adjustment of the voice signal by the specific terminal equipment can meet the following formula:

g _out ＝g _in (1-A _a )

wherein g _out G, outputting gain g after gain adjustment for the voice signal by the terminal equipment _in Input gain before gain adjustment for voice signal by terminal equipment, A _a Is the attenuation coefficient.

Wherein the attenuation coefficient A _a The following formula may be satisfied:

where f is the frequency of the first audio signal (unit: hz), d is the distance information between the terminal device and the user (unit: m),is air humidity.

In the embodiment of the application, the air humidity may be assigned differently by locating the current geographical location of the terminal device. For example, if the terminal device is close to coastal areas, the terminal device may assign a relatively high value to the air humidity, such as 60% or 70% or the like, due to the relatively high air humidity. If the terminal device is located close to an inland area, the terminal device may assign a relatively low value to the air humidity, such as 20% or 30% due to the relatively low air humidity. The current air humidity can also be obtained through terminal equipment networking, and detection and acquisition can also be carried out through a terminal equipment built-in hygrometer, and the mode of specifically obtaining the air humidity is not limited by the embodiment of the application.

It will be appreciated that the attenuation coefficient A _a The coefficient 7.4 in the calculation formula of (c) may be a varying value and the attenuation coefficient may be related to the air temperature as the audio signal propagates in air. The coefficient of 7.4 may be, for example, the attenuation coefficient A at an air temperature of about 20deg.C _a The corresponding coefficient value. Since the room temperature of the user typically floats around 20 ℃, it can also be understood that the coefficient 7.4 is the attenuation coefficient A _a Default values in the calculation formula of (c). The air temperature can also be obtained through terminal equipment networking to the current air temperature, can also be obtained through terminal equipment built-in thermometer detects, specifically obtains the mode of air humidity, and the embodiment of the application is not limited.

According to attenuation coefficient A _a As can be seen from the calculation formula of (a), the air humidity and the air temperature can influence the attenuation coefficient A _a At the same time, the human voice signals with different frequencies can obtain different attenuation coefficients A _a . Assuming that the air humidity and air temperature are relatively constant and that the terminal device can acquire the frequency of the input voice signalRate, attenuation coefficient A _a May be related to distance information between the terminal device and the user. Thus, the attenuation coefficient A is adjusted according to different distance information _a The gain of the input voice signal can be further adjusted, so that the adjusted audio signal output by the terminal equipment is more in line with the user experience when the user position changes.

In one possible implementation, the peak filter may be used in the middle-high frequency compensation module to perform gain adjustment on the voice signal through different attenuation coefficients a _a The gain of the human voice signal can be compensated or attenuated, so that the control of the frequency band and the bandwidth of the human voice signal is realized. Wherein the parameters of the peak filter may include one or more of: sample rate (sampling frequency, sfreq), center frequency (frequency central, fc), quality factor (Q), gain (gain, g), or order of the filter, etc. By way of example, the sampling rate may take on values of 48000Hz, 44100Hz, 16000Hz, etc.; the center frequency can be 1000Hz, 2000Hz, 3000Hz, 4000Hz, 6000Hz, etc.; the quality factors can be 0.5, 1.0, 1.414, 2.0, 3.0, 4.0, etc.; the gain g can be within the range of [ -10dB,10dB]、[-20dB,20dB]Etc.; the order of the filter may be 1 order, 2 order, 3 order, etc., and it is understood that the higher the order of the filter, the better the filtering effect.

Illustratively, fig. 10 shows a schematic diagram of gain adjustment of a human voice signal by a peak filter. Wherein the abscissa in the figure is frequency (unit: hz) and the ordinate is gain (unit: dB). It will be appreciated that the figure may be represented as a schematic diagram of a terminal device performing gain adjustment for two frequency points around 500Hz and around 1100 Hz. Taking a set of peak curves corresponding to 500Hz as an example, assuming that the gain of the audio signal at the time of input is 0dB, the highest one of the convex curves may correspond to compensating the gain of the signal by 12dB, the second highest one of the convex curves may correspond to compensating the gain of the signal by 10dB, and so on, the lower convex curves may be compensating the gain of the signal by 8dB, 6dB, 4dB, and 2dB, respectively. Further, the lowest one of the concave curves may correspond to attenuating the gain of the signal by 12dB, the second lowest concave curve may correspond to attenuating the gain of the signal by 10dB, and so on, the upper concave curves may be attenuating the gain of the signal by 8dB, 6dB, 4dB, and 2dB, respectively.

In the embodiment of the application, the terminal equipment adjusts the gain of the output audio signal according to the distance between the terminal equipment and the user, so that the user has better sound field experience in scenes such as film watching.

Optionally, on the basis of the embodiment corresponding to fig. 9, the performing, by the terminal device, horizontal offset calibration of the sound field on the mixed signal according to the horizontal direction included angle may include: the terminal equipment determines a directivity function in the horizontal direction according to the included angle in the horizontal direction; and the terminal equipment carries out horizontal offset calibration of the sound field on the mixed sound signal according to the directivity function in the horizontal direction.

In this embodiment of the application, after the terminal device obtains the signal of the horizontal direction contained angle based on the camera or other sensors, the signal of the horizontal direction contained angle can be transmitted to the sound field horizontal offset calibration module, and then, the sound field horizontal offset calibration module can calibrate the horizontal direction of the sound field for the input audio signal.

In the sound field calibration, the number of speakers, the positions of the speakers, and the like may be related.

As shown in fig. 11, a speaker is taken as an example of a point sound source, where the speaker may be located on a straight line in the terminal device, the distance between the speakers may be d, and the distance between the speaker and the point a in the sound field may be r _k The included angle between the loudspeaker and the horizontal direction of the point a in the sound field may be α. If n point sound sources with a distance d are arranged on a straight line, any one point sound source is arranged at a distance r from the point sound source _k The sound pressure generated at point a can satisfy the following formula:

wherein P is _0k The sensitivity of the kth point sound source, phi _k Can be the phase of the kth point sound source, lambda is the terminal equipmentWavelength r of sound wave radiated by the middle loudspeaker _k The distance of the kth point sound source from the a point in the sound field may be.

It will be appreciated that the terminal device may obtain the sensitivity of the point source based on the speakers, and the sensitivity corresponding to the different speakers may be the same or different. The value of the specific sensitivity is not limited in the embodiment of the present application. Based on the correlation principle between parameters such as sensitivity of a loudspeaker and a point sound source, the embodiment of the application can be combined with the loudspeaker to calibrate the sound field.

As shown in fig. 12, fig. 12 shows a schematic diagram of the array placement of horizontal sound field calibration speakers in a terminal device. Illustratively, taking the example that the horizontal sound field calibration speaker of the terminal device includes a first speaker and a second speaker, as shown by a or b in fig. 12, the first speaker and the second speaker may be located at the bottom of the terminal device, and the first speaker and the second speaker may be maintained on the same horizontal line. It can be understood that the speaker calibrated by the horizontal sound field in the terminal device may adopt a forward sound output mode, a downward sound output mode, or a lateral sound output mode, and the sound output mode of the speaker is not limited in the embodiments of the present application. Wherein. The forward sound output mode has a relatively good sound output effect compared with the downward sound output mode or the lateral sound output mode. In addition, in the terminal device, the horizontal sound field calibration may have 2 or more speakers, and it is understood that the greater the number of speakers, the better the effect of horizontal sound field calibration, and the number of speakers for horizontal sound field calibration is not limited in the embodiment of the present application.

If the phases and intensities of the point sound sources are identical in the linear array of the point sound sources, then the directivity function of the linear array of the point sound sourcesThe following formula may be satisfied:

wherein alpha is ₁ Can be a loudspeaker andhorizontal included angle of A point in sound field, d ₁ The distance between the first speaker and the second speaker may be, n may be the number of speakers in the terminal device, and λ may be the wavelength of the sound wave radiated by the speakers in the terminal device.

From the above formula, the directivity function of the sound fieldMay be related to the number of point sources in the sound field array, the distance between the point sources, which may also be referred to as array elements, and the wavelength of the sound waves radiated by the loudspeakers. When these parameters are all determined, the directivity function of the sound field may be at an angle alpha to the horizontal direction of the loudspeaker and point A in the sound field ₁ Related to the following. At this time, the directivity function of the sound field +.>The following formula may be satisfied:

from the above formula, it can be seen that if α is changed ₁ The directivity of the sound field can be adjusted.

Fig. 13 shows a simulation of the beamforming of point sound sources of different frequencies, where the beamformed directivity of the loudspeaker exhibits a fan-shaped distribution at a frequency of 500Hz, covering a relatively large range, where the beam has no apparent directivity. When the frequency is 1000Hz, the directivity of the beam forming of the speaker exhibits an elliptical distribution, and the coverage range becomes relatively smaller than that of the frequency of 500Hz, and at this time, the directivity is relatively remarkable. When the frequency is 2000Hz, the range covered by the sound field directivity of the speaker is relatively smaller than that when the frequency is 1000Hz, and at this time, the beam forming of the speaker has obvious directivity. As can be seen from the figure, the larger the frequency, the more pronounced the directivity of the sound source beamforming at that frequency point.

FIG. 14 shows beams from 0 to 180 degrees in the horizontal directionA simulated schematic diagram of the deflection therebetween. As can be seen from the figure, the change α ₁ The beam of the sound field can be deflected between 0 and 180 degrees in the horizontal direction, so that the calibration of the sound field in the horizontal direction can be realized.

Optionally, on the basis of the embodiment corresponding to fig. 9, the performing, by the terminal device, vertical offset calibration of the sound field on the mixed signal according to the vertical direction included angle may include: the terminal equipment determines a directivity function in the vertical direction according to the included angle in the vertical direction; and the terminal equipment performs vertical offset calibration of the sound field on the mixed signal according to the directivity function in the vertical direction.

In this embodiment of the present application, after the terminal device obtains the signal of the vertical direction included angle based on the camera or other sensors, the signal of the vertical direction included angle may be transmitted to the sound field vertical offset calibration module, and further, the sound field vertical offset calibration module may calibrate the vertical direction of the sound field for the input audio signal.

As shown in fig. 12, fig. 12 shows a schematic diagram of the array placement of vertical sound field calibration speakers in a terminal device. Illustratively, taking the example that the vertical sound field calibration speaker in the terminal device includes a third speaker and a fourth speaker, as shown by a in fig. 12, the third speaker and the fourth speaker may be located at the top of the terminal device, and the third speaker and the fourth speaker may be maintained on the same horizontal line. It can be understood that the speaker for calibrating the vertical sound field in the terminal device may adopt a forward sound output mode, an upward sound output mode, or as shown in b in fig. 12, the speaker may adopt a lateral sound output mode, and the sound output mode of the speaker is not limited. The forward sound output mode has better sound output effect than the upward sound output mode or the lateral sound output mode. In addition, in the terminal device, the number of the vertical sound field calibration may be 2 or more speakers, and it is understood that the greater the number of the speakers, the better the effect of the vertical sound field calibration, and the number of the vertical sound field calibration speakers is not limited in the embodiment of the present application.

If, in a linear array of point sources, the phases of the individual point sources are identical, of equal intensity,then the directivity function of the linear array of point sourcesThe following formula may be satisfied:

wherein alpha is ₂ Can be the included angle d of the vertical direction of the loudspeaker and the point A in the sound field ₂ The distance between the third speaker and the fourth speaker may be, n may be the number of speakers in the terminal device, and λ may be the wavelength of the sound wave radiated by the speakers in the terminal device.

From the above formula, the directivity function of the sound fieldMay be related to the number of point sources in the sound field array, the distance between the point sources, which may also be referred to as array elements, and the wavelength of the sound waves radiated by the loudspeakers. When these parameters are all determined, the directivity function of the sound field array +.>Can form an included angle alpha with the vertical direction of the loudspeaker and the point A in the sound field ₂ Related to the following. At this time, the directivity function of the sound field +.>The following formula may be satisfied:

from the above formula, it can be seen that if α is changed ₂ The directivity of the sound field can be adjusted. That is, change α ₂ The value can deflect the wave beam of the sound field between 0 and 180 degrees in the vertical direction, so that the calibration of the sound field in the vertical direction can be realized.

In the embodiment of the application, the terminal equipment correspondingly adjusts the directivity of the sound field based on the horizontal direction included angle and the vertical direction included angle between the terminal equipment and the user, so that the calibration of the sound field in the horizontal direction and the vertical direction is realized, and the user has better sound field experience in the scenes of watching and the like.

Optionally, on the basis of the embodiment corresponding to fig. 9, the terminal device obtains information between the user and the terminal device, which may include: the terminal equipment obtains information between the user and the terminal equipment based on the camera.

In the embodiment of the application, the terminal equipment can obtain the information between the user and the terminal equipment based on the camera, so that the real-time calibration of the sound field can be realized according to the position information of the user and the linkage of the audio algorithm module, the position of the sound field is positioned near the center position of the terminal equipment, and the user experience is improved.

The foregoing description of the solution provided in the embodiments of the present application has been mainly presented in terms of a method. The following describes an apparatus for performing the above method provided in the embodiments of the present application. As shown in fig. 14, fig. 14 is a schematic structural diagram of a sound field calibration device provided in an embodiment of the present application, where the sound field calibration device may be a terminal device in an embodiment of the present application, or may be a chip or a chip system in the terminal device.

As shown in fig. 15, the sound field calibration apparatus 1500 may be used in a communication device, a circuit, a hardware component, or a chip, and includes: a processing unit 1501. Wherein the processing unit 1501 is used to support the sound field calibration apparatus 1500 to perform the steps of information processing.

Specifically, the embodiment of the present application provides a sound field calibration apparatus 1500, where a terminal device may include: a processing unit 1501 for acquiring information between a user and a terminal device; according to the information, the method can be used for carrying out voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal equipment to obtain a second audio signal; and playing the second audio signal, wherein the distance between the sound field position of the second audio signal and the center position of the terminal device is smaller than the first preset value. The processing unit 1501 may be further configured to perform voice extraction on the first audio signal to obtain a voice signal and a background sound signal; performing voice adjustment on the voice signal according to the distance information to obtain a compensated voice signal; mixing the background sound signal and the compensated human sound signal to obtain a mixed sound signal; the method can also be used for carrying out horizontal offset calibration of the sound field on the mixed signal according to the included angle in the horizontal direction and carrying out vertical offset calibration of the sound field on the mixed signal according to the included angle in the vertical direction. A processing unit 1501, which may be configured to determine an attenuation coefficient from the distance information; and can also be used for gain compensation or attenuation of the human voice signal according to the attenuation coefficient. A processing unit 1501, configured to determine a directivity function in a horizontal direction according to the horizontal direction included angle; the method can be used for calibrating the horizontal offset of the sound field of the mixed signal according to the directivity function in the horizontal direction. A processing unit 1501, configured to determine a directivity function in a vertical direction according to the vertical direction included angle; the method can be used for calibrating the vertical offset of the sound field of the mixed signal according to the directivity function in the vertical direction.

In a possible implementation manner, the sound field calibration apparatus 1500 may also include: a communication unit 1502. Specifically, the communication unit is configured to support the sound field calibration apparatus 1500 to perform the steps of transmitting data and receiving data. The communication unit 1502 may be an input or output interface, a pin or circuit, or the like.

In a possible implementation manner, the sound field calibration device may further include: and a memory cell 1503. The processing unit 1501 and the storage unit 1503 may be connected by a line. The memory unit 1503 may include one or more memories, which may be one or more devices, circuits, or means for storing programs or data. The storage unit 1503 may exist independently and be connected to the processing unit 1501 provided in the sound field calibration apparatus through a communication line. The memory unit 1503 may also be integrated with the processing unit 1501.

The storage unit 1503 may store computer-executable instructions of the method in the terminal device to cause the processing unit 1501 to perform the method in the above-described embodiment. The storage unit 1503 may be a register, a cache, a RAM, or the like, and the storage unit 1503 may be integrated with the processing unit 1501. The storage unit 1503 may be a read-only memory (ROM) or other type of static storage device that may store static information and instructions, and the storage unit 1503 may also be independent of the processing unit 1501.

Fig. 16 is a schematic hardware structure of another terminal device according to an embodiment of the present application, as shown in fig. 16, where the terminal device includes a processor 1601, a communication line 1604 and at least one communication interface (illustrated in fig. 16 by taking a communication interface 1603 as an example).

The processor 1601 may be a general purpose central processing unit (central processing unit, CPU), microprocessor, application Specific Integrated Circuit (ASIC), or one or more integrated circuits for controlling the execution of the programs of the present application.

Communication line 1604 may include circuitry for communicating information between the components described above.

Communication interface 1603, using any transceiver-like device, is used to communicate with other devices or communication networks, such as ethernet, wireless local area network (wireless local area networks, WLAN), etc.

Possibly, the terminal device may also comprise a memory 1602.

Memory 1602 may be, but is not limited to, read-only memory (ROM) or other type of static storage device that can store static information and instructions, random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, as well as electrically erasable programmable read-only memory (EEPROM), compact disc-read only memory (compact disc read-only memory) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be self-contained and coupled to the processor via communication line 1604. The memory may also be integrated with the processor.

The memory 1602 may be used for storing computer-executable instructions for executing aspects of the present application, and is controlled for execution by the processor 1601. The processor 1601 may be configured to execute computer-executable instructions stored in the memory 1602 to implement the methods provided by embodiments of the present application.

Possibly, the computer-executed instructions in the embodiments of the present application may also be referred to as application program code, which is not specifically limited in the embodiments of the present application.

In a particular implementation, as one embodiment, the processor 1601 may include one or more CPUs, such as CPU0 and CPU1 in fig. 16.

In a specific implementation, as an embodiment, the terminal device may include a plurality of processors, such as processor 1601 and processor 1605 in fig. 16. Each of these processors may be a single-core (single-CPU) processor or may be a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

Embodiments of the present application also provide a computer program product comprising one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL), or wireless (e.g., infrared, wireless, microwave, etc.), or semiconductor medium (e.g., solid state disk, SSD)) or the like.

Embodiments of the present application also provide a computer-readable storage medium. The methods described in the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer readable media can include computer storage media and communication media and can include any medium that can transfer a computer program from one place to another. The storage media may be any target media that is accessible by a computer.

As one possible design, the computer-readable medium may include compact disk read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disk memory; the computer readable medium may include disk storage or other disk storage devices. Moreover, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, digital versatile disc (digital versatile disc, DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

Embodiments of the present application are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processing unit of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Claims

1. A method of sound field calibration, the method comprising:

the terminal equipment acquires information between a user and the terminal equipment, wherein the information comprises the following components: the method comprises the steps of enabling distance information between the terminal equipment and a user, enabling an included angle between the terminal equipment and the horizontal direction of the user and enabling an included angle between the terminal equipment and the vertical direction of the user to be the same;

The terminal equipment carries out voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal equipment according to the information to obtain a second audio signal;

the terminal equipment plays the second audio signal, wherein the distance between the sound field position of the second audio signal and the central position of the terminal equipment is smaller than a first preset value;

the terminal equipment carries out voice adjustment, horizontal sound field calibration and vertical sound field calibration on a first audio signal to be played by the terminal equipment according to the information to obtain a second audio signal, and the method comprises the following steps:

the terminal equipment performs voice extraction on the first audio signal to obtain a voice signal and a background sound signal;

the terminal equipment adjusts the voice of the voice signal according to the distance information to obtain a compensated voice signal;

the terminal equipment mixes the background sound signal and the compensated voice signal to obtain a mixed sound signal;

the terminal equipment determines a directivity function of the horizontal direction according to the included angle of the horizontal direction;

the terminal equipment carries out horizontal offset calibration of the sound field on the sound mixing signal according to the directivity function in the horizontal direction;

The terminal equipment determines a directivity function in the vertical direction according to the included angle in the vertical direction;

and the terminal equipment carries out vertical offset calibration of the sound field on the sound mixing signal according to the directivity function in the vertical direction.

2. The method of claim 1, wherein the terminal device performs voice adjustment on the voice signal according to the distance information to obtain a compensated voice signal, and the method comprises:

the terminal equipment determines an attenuation coefficient according to the distance information;

when the distance information is larger than or equal to a second preset value, the terminal equipment performs gain compensation on the voice signal according to the attenuation coefficient;

and when the distance information is smaller than the second preset value, the terminal equipment carries out gain attenuation on the voice signal according to the attenuation coefficient.

3. The method of claim 2, wherein the attenuation coefficient satisfies the following formula:

wherein A is _a For the attenuation coefficient, f is the frequency of the first audio signal, d is the distance information,is relative humidity.

4. The method of claim 1, wherein the terminal device comprises a first speaker, a second speaker, and wherein the directivity function in the horizontal direction satisfies the following formula:

Wherein,as a directivity function of the horizontal direction, alpha ₁ D is the included angle of the horizontal direction ₁ And n is the number of the speakers in the terminal equipment, and lambda is the wavelength of sound waves radiated by the speakers in the terminal equipment.

5. The method of claim 4, wherein the terminal device comprises a first speaker, a second speaker, comprising:

the first speaker and the second speaker are located at the bottom of the terminal device, and the first speaker and the second speaker are maintained on the same horizontal line.

6. The method of claim 1, wherein the terminal device comprises a third speaker, a fourth speaker, and wherein the directivity function in the vertical direction satisfies the following formula:

wherein,as a directivity function of the vertical direction, alpha ₂ D is the included angle of the vertical direction ₂ And n is the number of the speakers in the terminal equipment, and lambda is the wavelength of sound waves radiated by the speakers in the terminal equipment.

7. The method of claim 6, wherein the terminal device comprises a third speaker, a fourth speaker, comprising:

The third speaker and the fourth speaker are located at the top of the terminal device, and the third speaker and the fourth speaker remain on the same horizontal line.

8. The method according to any of claims 1-7, wherein the terminal device obtaining information between a user and the terminal device comprises:

the terminal equipment acquires information between the user and the terminal equipment based on a camera.

9. A terminal device, comprising: a memory for storing a computer program and a processor for executing the computer program to perform the sound field calibration method according to any one of claims 1-8.

10. A computer-readable storage medium storing instructions that, when executed, cause a computer to perform the sound field calibration method of any one of claims 1-8.

11. A computer program product comprising a computer program which, when run, causes a terminal device to perform the sound field calibration method according to any one of claims 1-8.