WO2023070778A1 - Audio output control method and system, and related assemblies - Google Patents

Abstract

Disclosed in the present application are an audio output control method and system, and a head-mounted device and a computer-readable storage medium. The audio output control method comprises: acquiring an initial audio signal of a virtual sound source in the current environment where a head-mounted device is located; calculating a phase modulation parameter on the basis of the current head contour parameter of a wearer of the head-mounted device; calculating an amplitude modulation parameter on the basis of the position of the virtual sound source relative to the head-mounted device; and controlling a sound production apparatus of the head-mounted device to perform stereoscopic sound effect modulation on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter, and outputting the modulated audio signal. In the present application, the sense of space on a horizontal plane is reconstructed by using amplitude translation, and the sense of space in a vertical direction is enhanced by using phase modulation, so that a wearer can sense the real orientation of a sound source better, and production of a positioning effect in the sound source is avoided, thereby improving the sound field sense of space and the user experience of a head-mounted device.

Description

An audio output control method, system and related components

This application claims the priority of the Chinese patent application with the application number 202111276794.0 and the title of the invention "An Audio Output Control Method, System and Related Components" filed with the China Patent Office on October 29, 2021, the entire contents of which are incorporated by reference in this application.

technical field

The present application relates to the field of head-mounted devices, in particular to an audio output control method, system and related components.

Background technique

VR (Virtual Reality, virtual reality) products use computer technology to simulate a three-dimensional virtual world to create an immersive experience for users. VR products have three core features: immersion, interactivity, and imagination. Immersion is the most basic feature of VR products, and sound effects are one of the important indicators for perfect immersion in VR products. At present, most of the VR products on the market are dual-speaker. The two speakers sound at the left and right ears. The amplitude translation of the two speakers is distributed, so that the two speakers can sound according to the allocated amplitude, so as to achieve the purpose of sound image reconstruction. However, this simple amplitude translation scheme makes the user's perception of the direction of the sound source relatively simple when using it, and it is easy to produce a localization effect in the sound source. The sense of space in the sound field is poor, and the acoustic immersion effect of the product is poor.

Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present.

Contents of the invention

The purpose of this application is to provide an audio output control method and system, a head-mounted device and a computer-readable storage medium, which uses amplitude translation to reconstruct the sense of space on the horizontal plane, and uses phase modulation to enhance the sense of space in the vertical direction. It enables the wearer to better perceive the real orientation of the sound source, avoids the localization effect of the sound source, and improves the sound field space and user experience of the headset.

In order to solve the above technical problems, the application provides an audio output control method, including:

Obtain the initial audio signal of the virtual sound source in the current scene where the headset is located;

calculating phase modulation parameters based on current head profile parameters of the wearer of the head-mounted device;

calculating an amplitude modulation parameter based on a position of the virtual sound source relative to the head-mounted device;

Controlling the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.

Optionally, the phase modulation parameter includes a head-related transfer function.

Optionally, the head-mounted device includes n sound emitting devices, where n is an integer greater than 2;

The audio output control method also includes:

determining a to-be-operated sound emitting device among all the sound emitting devices based on the position of the virtual sound source relative to the head-mounted device;

The process of controlling the sounding device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal includes:

Controlling the standby sound generating device in the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.

Optionally, a sound image control interval is reconstructed for every two adjacent sound emitting devices;

The process of determining the sound emitting device to be operated among all the sound emitting devices based on the position of the virtual sound source relative to the head-mounted device includes:

obtaining an incident direction of the virtual sound source based on a position of the virtual sound source relative to the head-mounted device;

determining the target sound image control interval where the incident direction is located in all the sound image control intervals;

Selecting a to-be-worked sound emitting device among the sound emitting devices reconstructing the target sound image control interval.

Optionally, the n sound emitting devices are located on the same horizontal plane, and the line connecting the n sound emitting devices and the midpoint of the human head on the horizontal plane divides the horizontal plane into n sound image control intervals;

The process of determining the target sound image control interval where the incident direction is located in all the sound image control intervals includes:

Obtain the projection of the incident direction on the horizontal plane;

The sound image control interval where the projection is located is determined as a target sound image control interval.

Optionally, the process of selecting a sounding device to be worked among the sounding devices in the reconstruction of the target sound image control interval includes:

judging whether the projection is on any of the lines dividing the audio-image control interval;

If yes, among the two sound emitting devices reconstructing the target sound image control interval, the sound emitting device corresponding to the line where the projection is located is used as the sound emitting device to be operated;

If not, the two sound emitting devices reconstructing the target sound image control interval are both used as the waiting sound emitting devices.

Optionally, the process of calculating the amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device includes:

determining and reconstructing the bisector of the angle between the two sounding devices in the target sound image control interval based on the position of the midpoint of the human head;

Calculate the respective amplitude modulation parameters of the two sounding devices according to the first relational expression, the first relational expression is

in,

is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.

In order to solve the above technical problems, the present application also provides an audio output control system, including:

An acquisition module, configured to acquire an initial audio signal of a virtual sound source in the current scene where the head-mounted device is located;

A first calculation module, configured to calculate a phase modulation parameter based on the current head profile parameters of the wearer of the head-mounted device;

A second calculation module, configured to calculate an amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device;

The output control module is used to control the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.

In order to solve the above technical problems, the present application also provides a head-mounted device, including:

memory for storing computer programs;

A processor, configured to implement the steps of the audio output control method described in any one of the above when executing the computer program.

In order to solve the above-mentioned technical problems, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it realizes any of the above-mentioned The steps of the audio output control method. This application provides an audio output control method, which calculates the amplitude modulation parameters of the sound emitting device according to the position of the virtual sound source relative to the head-mounted device, and at the same time obtains the current head contour parameters of the wearer of the head-mounted device, and calculates the sound based on the head contour parameters The phase modulation parameters of the device, and then control the sounding device to modulate the stereo sound effect on the initial audio signal of the virtual sound source according to the amplitude modulation parameter and the phase modulation parameter, and then output it. This application has considered the size of the sound amplitude output by the sounding device. Construct the sense of space of the virtual sound source on the horizontal plane, and also supplement and correct the phase of the virtual sound source with amplitude translation according to the current head contour parameters, so as to reconstruct the sense of space of the virtual sound source in the vertical direction. The audio signal output by the device can enable the wearer to better perceive the real orientation of the sound source, avoid the localization effect of the sound source, and improve the spatial sense of the sound field and user experience of the head-mounted device. The present application also provides an audio output control system, a head-mounted device, and a computer-readable storage medium, which have the same beneficial effect as the above-mentioned audio output control method.

Description of drawings

In order to illustrate the embodiments of the present application more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. As far as people are concerned, other drawings can also be obtained based on these drawings on the premise of not paying creative work.

Fig. 1 is a flow chart of the steps of an audio output control method provided by the present application;

FIG. 2 is a schematic diagram of a phase modulation effect provided by the present application;

FIG. 3 is a schematic structural diagram of a head-mounted device provided by the present application;

Fig. 4 is a schematic layout diagram of a sounding device provided by the present application;

FIG. 5 is a schematic diagram of an audio-image control interval provided by the present application;

FIG. 6 is a schematic diagram of a tangent law amplitude translation provided by the present application;

Fig. 7 is a comparison diagram of the effect of amplitude translation binaural sound intensity difference provided by the present application;

FIG. 8 is a schematic structural diagram of an audio output control system provided by the present application.

Detailed ways

The core of this application is to provide an audio output control method and system, a head-mounted device and a computer-readable storage medium, which uses amplitude translation to reconstruct the sense of space on the horizontal plane, and uses phase modulation to enhance the sense of space in the vertical direction. It enables the wearer to better perceive the real orientation of the sound source, avoids the localization effect of the sound source, and improves the sound field space and user experience of the headset.

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

Please refer to Figure 1, Figure 1 is a flow chart of the steps of an audio output control method provided by the present application, the audio output control method includes:

S101: Obtain an initial audio signal of a virtual sound source in the current scene where the headset is located;

Among them, the head-mounted device can specifically be a head-mounted VR device. The VR device can use a computer to simulate a three-dimensional virtual scene. Users can experience VR movies or VR games through the VR device. In different VR movies or VR games, VR will be required. The device provides corresponding sound effects for the user, so that the user can be immersed in the virtual world.

Specifically, the head-mounted device in this example includes a sound generating device, which is used to output the audio signal of the virtual sound source in the current three-dimensional virtual scene, so as to provide the user with the sound effect of a certain VR movie or a certain VR game in the current scene, Specifically, the sound generating device may be a loudspeaker. After obtaining the initial audio signal of the virtual sound source, the sound generating device will modulate the initial audio signal according to the set modulation parameters before outputting, where the initial audio signal is the unprocessed audio signal of the virtual sound source.

It can be understood that according to the position of the wearer in the current scene of the VR movie or VR game, the virtual sound source that needs to be sounded can be determined currently, and the number of virtual sound sources in a scene can be one or more indivual.

S102: Calculate the phase modulation parameter based on the current head profile parameter of the wearer of the head-mounted device;

Considering that in order to make the sound effect brought by the head-mounted device to the user to have a sense of space and provide the user with a relatively high auditory experience, it is usually used to reconstruct the sound image of the two sounding devices in the head-mounted device by means of amplitude translation , but the spatial sense of the sound source in the vertical direction cannot be accurately reconstructed only by amplitude translation, so this step also collects the current head contour parameters of the wearer of the head-mounted device, and the phase modulation parameters can be calculated based on the current head contour parameters, The phase modulation parameter here specifically refers to the head-related transfer function of the wearer, and the phase modulation of the audio signal is supplemented by the head-related transfer function to further enhance the sense of use of spatial audio. The effect of phase modulation is shown in Figure 2. It can be seen that the phase spectrum of the audio signal after phase modulation is closer to the phase spectrum of the original sound source than the phase spectrum of the audio signal originally emitted by the sounding device.

Specifically, the head-mounted device is also equipped with an infrared scanner for collecting the wearer's current head contour parameters. Two infrared scanners can be set up, arranged on both sides of the head respectively. The structure diagram of the head-mounted device can be referred to in Figure 3 Figure 3 only shows the layout of the infrared scanner ① and the sounding device ② on one side of the head. The current head contour parameters include head contour parameters and ear contour parameters. It can be understood that, considering that the wearer of the head-mounted device is not fixed, the current head profile parameters of the current wearer can be acquired each time the head-mounted device is used, so as to obtain the head shape corresponding to the current wearer. related transfer functions.

Among them, the relational expression of the head-related transfer function is

H _L and H _R represent the head-related transfer function corresponding to the left ear of the current wearer and the head-related transfer function corresponding to the right ear respectively, p _L and p _R represent the audio signal received by the left ear and the audio signal received by the right ear respectively , p ₀ represents the audio signal at the center of the head when the head is not present. In general, _HL and _HR are functions of the horizontal azimuth angle θ, the elevation angle φ of the virtual sound source, the distance r from the virtual sound source to the center of the head, and the sound wave parameter f.

S103: Calculate an amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device;

S104: Control the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter, and output the modulated audio signal.

Specifically, after obtaining the position of the virtual sound source relative to the head-mounted device, calculate the amplitude modulation parameter of the sound generating device, control the sound generating device to modulate the amplitude of the initial audio signal according to the amplitude modulation parameter calculated in S103, and control the sound generating device The phase of the initial audio signal is modulated according to the phase modulation parameter calculated in S102, and then the modulated audio signal is output, so that the user can better perceive the orientation of the virtual sound source and improve the acoustic immersion of the sound emitting device. As an optional embodiment, the amplitude of the original audio signal may be modulated first, and then the phase of the amplitude-modulated audio signal may be modulated. It can be understood that when phase modulation is performed, the modulated audio signal can be obtained by inputting the corresponding audio signal into the head-related transfer function. In this embodiment, the execution sequence of S101-S103 is not limited.

It can be seen that in this embodiment, the amplitude modulation parameters of the sounding device are calculated according to the position of the virtual sound source relative to the head-mounted device, and at the same time, the current head contour parameters of the wearer of the head-mounted device are obtained, and the phase modulation of the sounding device is calculated based on the head contour parameters parameter, and then control the sounding device to modulate the stereo sound effect on the initial audio signal of the virtual sound source according to the amplitude modulation parameter and the phase modulation parameter, and then output it. This application not only considers the magnitude of the sound output by the sounding device, but also reconstructs the virtual sound source The sense of space on the horizontal plane also supplements and corrects the phase of the virtual sound source with amplitude translation according to the current head contour parameters, so as to reconstruct the sense of space of the virtual sound source in the vertical direction. Therefore, the audio output by the sounding device of the present application The signal can enable the wearer to better perceive the real orientation of the sound source, avoid the localization effect of the sound source, and improve the spatial sense of the sound field and user experience of the headset.

On the basis of above-mentioned embodiment:

As an optional embodiment, the head-mounted device includes n sound emitting devices, where n is an integer greater than 2;

The audio output control method also includes:

determining the sound emitting device to be operated among all sound emitting devices based on the position of the virtual sound source relative to the head-mounted device;

The process of controlling the sounding device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal includes:

Controlling the ready-to-work sound emitting device in the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter, and output the modulated audio signal.

In the prior art, in order to construct the spatial sense of the sound field, a sound emitting device is generally installed on the head-mounted device near the two ears, and after the audio signal corresponding to the virtual sound source is obtained, the two sound generating devices Set the modulation parameters so that the two sound-generating devices respectively modulate and output the received audio signals to achieve the purpose of reconstructing the sound image. However, this scheme of setting up two sound emitting devices will make the user's perception of the direction and position of the sound source relatively simple when using the head-mounted device, and it is easy to produce a localization effect in the sound source, making the sound field spatial sense felt by the user relatively small. Difference. Based on this, the head-mounted device provided in this embodiment is provided with at least three sound generating devices for reconstructing the sound image. In order to further enhance the spatial sense of the sound field, this embodiment first selects the sound emitting device in the head-mounted device according to the position of the virtual sound source relative to the head-mounted device, and the sound-emitting device to be operated is modulated according to the calculated phase modulation parameters and amplitude The parameters modulate the stereo sound effect on the initial audio signal of the virtual sound source, and output the modulated audio signal. The sound image reconstructed by the working sound generating device can improve the user's perception of the direction of the virtual sound source. Wherein, there may be one or two sound emitting devices to be operated, which can be selected according to the position of the virtual sound source relative to the head-mounted device, which is not limited in this application.

As an optional embodiment, every two adjacent sounding devices reconstruct a sound image control interval;

The process of determining the sound emitting device to work among all the sound emitting devices based on the position of the virtual sound source relative to the headset includes:

The incident direction of the virtual sound source is obtained based on the position of the virtual sound source relative to the head-mounted device;

Determining the target sound image control interval where the incident direction is located in all sound image control intervals;

Select the to-be-worked sounding device among the sounding devices in the reconstruction target sound image control section.

As an optional embodiment, the n sounding devices are located on the same horizontal plane, and the line connecting the n sounding devices and the midpoint of the human head on the horizontal plane divides the horizontal plane into n sound image control intervals;

The process of determining the target panning interval in which the incident direction is located among all panning intervals includes:

Obtain the projection of the incident direction on the horizontal plane;

The sound image control interval where the projection is located is determined as the target sound image control interval.

Specifically, this embodiment will be described by taking a head-mounted device provided with six sound emitting devices as an example. The sounding points of the six sounding devices are distributed symmetrically on both sides of the human head. In order to further improve the accuracy of the sound image reconstruction, in this embodiment, two adjacent sounding devices are selected to obtain the control interval for reconstructing the sound image, that is, the sound image like control intervals. The 6 sounding devices can be located on the same plane or on different planes. In order to facilitate the determination of which two sounding devices are responsible for the control interval of the virtual sound source, as a preferred embodiment, the 6 sounding devices can be set on the same plane. level. A coordinate system with the midpoint of the human head as the origin is established on this horizontal plane. It is understandable that the lines connecting the six sound emitting devices with the origin of the coordinate system can divide the horizontal plane into six sound-image control intervals, as shown in Figure 4 , the 6 sound generating devices are respectively marked as a, b, c, d, e, f, and the 6 sound-image control intervals are respectively marked as Ⅰ, Ⅱ, Ⅲ, Ⅳ, Ⅴ, Ⅵ.

It can be understood that, based on the position of the virtual sound source relative to the head-mounted device, the incident direction of the virtual sound source can be determined. Assuming that there is a virtual sound source in the current scene of a VR movie or VR game, the projection position of the incident direction of the virtual sound source on the horizontal plane is shown in Figure 5. The projection is located in the sound-image control interval VI, and the sound-image The control interval VI is the target audio image control interval, and the sound emitting device to be worked is determined from the two sound emitting devices responsible for the sound image control interval VI, that is, the sound emitting device to be worked is selected from the sound emitting device a and the sound emitting device d, so as to improve the The accuracy of sound image construction improves the sense of space in the sound field of the head-mounted device.

As an optional embodiment, the process of selecting a sound emitting device to be operated among the sound emitting devices in the reconstructed target sound image control interval includes:

Judging whether the projection is on any connection line dividing the audio-image control interval;

If so, among the two sound emitting devices in the reconstruction target sound image control section, the sound emitting device corresponding to the line where the projection is located is taken as the sound emitting device to be worked;

If not, the two sound emitting devices in the reconstructed target sound image control interval are both regarded as standby sound emitting devices.

Specifically, when selecting a sounding device to be operated, it is necessary to determine the orientation of the virtual sound source first, and determine whether the virtual sound source is between two sounding devices or overlaps with any sounding device. The sounding device that overlaps the virtual sound source can produce sound. Specifically, it can be judged whether the projection is on any connection that divides the audio-image control interval, and if the projection is on any connection that divides the audio-image control interval, then use the sounding device corresponding to the connection where it is located as the waiting to work Sounding device, if not, it means that the orientation of the virtual sound source is between the two sounding devices constituting the target sound image control section, and these two sounding devices are both regarded as the waiting sounding devices at this time.

Specifically, the angle between each sound emitting device and the target direction can be determined in advance. The target direction here can be the positive direction of the x-axis or the negative direction of the x-axis or the positive direction of the y-axis or the negative direction of the y-axis of the coordinate axis. When all the sounding devices are determined clockwise or counterclockwise, it is assumed that the angle between each sounding device and the positive direction of the y-axis is determined clockwise, that is, θa～θf. After determining the incident direction of the virtual sound source, determine the angle θr between the projection of the incident direction of the virtual sound source on the horizontal plane in the clockwise direction and the positive direction of the y-axis, and determine whether there is the same angle as the θr in θa～θf , assuming that θr and θa are the same, then the sounding device a is the waiting sounding device, if θd<θr<θa, then the sounding device d and the sounding device a are the waiting sounding devices.

As an optional embodiment, the process of calculating the amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device includes:

determining and reconstructing the bisector of the angle between the two sounding devices in the target sound image control interval based on the position of the midpoint of the human head;

Calculate the respective amplitude modulation parameters of the two sounding devices according to the first relational expression, the first relational expression is

in,

is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.

Specifically, after determining the two to-be-operated sound-emitting devices corresponding to the target sound image interval, the amplitude modulation parameters of the two to-be-operated sound-emitting devices can be allocated by means of amplitude translation. In order to avoid the influence of head rotation, the present application uses the first relational expression to calculate the amplitude modulation parameters of the two to-be-operated sound emitting devices. Referring to Figure 6, establish a coordinate system with the human head as the origin, and the two sounding devices to be worked are located on the circle centered at the listening point (the midpoint of the human head), and the angles of the two sounding devices to be worked are respectively

and

The angle between the virtual sound source and the bisector of the two sounding devices is

In this embodiment, a sound-generating device closer to the virtual sound source is used as the first sound-generating device, and the other sound-generating device is used as the second sound-generating device. Further, after calculating the amplitude modulation parameter g1 of the first sounding device and the amplitude modulation parameter g2 of the second sounding device respectively, the first sounding device and the second sounding device are respectively based on g1, g2 and the head-related transfer function. The initial audio signal of the virtual sound source is subjected to stereo sound modulation, and the modulated audio signals are respectively output.

The real sound source sends out the binaural amplitude difference received by the human ear from the direction of the arrow, and the binaural amplitude difference received by the human ear when the sound played by traditional VR is played by the ear, and the binaural amplitude difference received by the output sound according to the multi-speaker distribution amplitude. The comparison results of the ear amplitude difference are shown in Figure 7. It is not difficult to see that the binaural amplitude difference of the sound allocated by the amplitude translation is closer to the binaural amplitude difference of the original sound. In the case of VR playback, it is closer to the real position.

To sum up, this application increases the number of sounding devices on the VR device, and distributes the amplitude of the sound of different sounding devices with the amplitude translation of the tangent law, and uses infrared scanning to calculate the head transfer function to supplement the phase modulation of the speaker to achieve Control the human ear's perception of the position of the sound source, and improve the sense of space in the sound field of VR products.

Please refer to FIG. 8. FIG. 8 is a schematic structural diagram of an audio output control system provided by the present application. The audio output control system includes:

An acquisition module 11, configured to acquire an initial audio signal of a virtual sound source in the current scene where the head-mounted device is located;

The first calculation module 12 is used to calculate the phase modulation parameter based on the current head profile parameters of the wearer of the head-mounted device;

The second calculation module 13 is used to calculate the amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device;

The output control module 14 is used to control the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.

It can be seen that in this embodiment, the amplitude modulation parameters of the sounding device are calculated according to the position of the virtual sound source relative to the head-mounted device, and at the same time, the current head contour parameters of the wearer of the head-mounted device are obtained, and the phase modulation of the sounding device is calculated based on the head contour parameters parameter, and then control the sounding device to modulate the stereo sound effect on the initial audio signal of the virtual sound source according to the amplitude modulation parameter and the phase modulation parameter, and then output it. This application not only considers the magnitude of the sound output by the sounding device, but also reconstructs the virtual sound source The sense of space on the horizontal plane also supplements and corrects the phase of the virtual sound source with amplitude translation according to the current head contour parameters, so as to reconstruct the sense of space of the virtual sound source in the vertical direction. Therefore, the audio output by the sounding device of the present application The signal can enable the wearer to better perceive the real orientation of the sound source, avoid the localization effect of the sound source, and improve the spatial sense of the sound field and user experience of the headset.

As an optional embodiment, the phase modulation parameter includes a head-related transfer function.

As an optional embodiment, the head-mounted device includes n sound emitting devices, where n is an integer greater than 12;

The audio output control system also includes:

A determining module, configured to determine a sound emitting device to be operated among all sound emitting devices based on the position of the virtual sound source relative to the head-mounted device;

The process of controlling the sounding device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal includes:

Controlling the ready-to-work sound emitting device in the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter, and output the modulated audio signal.

As an optional embodiment, every two adjacent sounding devices reconstruct a sound image control interval;

The process of determining the sound emitting device to work among all the sound emitting devices based on the position of the virtual sound source relative to the headset includes:

The incident direction of the virtual sound source is obtained based on the position of the virtual sound source relative to the head-mounted device;

Determining the target sound image control interval where the incident direction is located in all sound image control intervals;

Select the to-be-worked sounding device among the sounding devices in the reconstruction target sound image control section.

As an optional embodiment, the n sounding devices are located on the same horizontal plane, and the line connecting the n sounding devices and the midpoint of the human head on the horizontal plane divides the horizontal plane into n sound image control intervals;

The process of determining the target panning interval in which the incident direction is located among all panning intervals includes:

Obtain the projection of the incident direction on the horizontal plane;

The sound image control interval where the projection is located is determined as the target sound image control interval.

As an optional embodiment, the process of selecting a sound emitting device to be operated among the sound emitting devices in the reconstructed target sound image control interval includes:

Judging whether the projection is on any connection line dividing the audio-image control interval;

If so, among the two sound emitting devices in the reconstruction target sound image control section, the sound emitting device corresponding to the line where the projection is located is taken as the sound emitting device to be worked;

If not, the two sound emitting devices in the reconstructed target sound image control interval are both regarded as standby sound emitting devices.

As an optional embodiment, the process of calculating the amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device includes:

determining and reconstructing the bisector of the angle between the two sounding devices in the target sound image control interval based on the position of the midpoint of the human head;

Calculate the respective amplitude modulation parameters of the two sounding devices according to the first relational expression, the first relational expression is

in,

is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.

In addition, the present application provides a head-mounted device, including:

memory for storing computer programs;

The processor is configured to implement the steps of the audio output control method described in any one of the above embodiments when executing the computer program.

Specifically, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The processor provides calculation and control capabilities for the head-mounted device. When executing the computer program stored in the memory, the following steps can be implemented: obtaining the initial audio signal of the virtual sound source in the current scene where the head-mounted device is located; Calculate the phase modulation parameter based on the current head contour parameters of the user; calculate the amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device; control the sounding device of the head-mounted device to perform stereo sound effects on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter Modulate and output the modulated audio signal.

It can be seen that in this embodiment, the amplitude modulation parameters of the sounding device are calculated according to the position of the virtual sound source relative to the head-mounted device, and at the same time, the current head contour parameters of the wearer of the head-mounted device are obtained, and the phase modulation of the sounding device is calculated based on the head contour parameters parameter, and then control the sounding device to modulate the stereo sound effect on the initial audio signal of the virtual sound source according to the amplitude modulation parameter and the phase modulation parameter, and then output it. This application not only considers the magnitude of the sound output by the sounding device, but also reconstructs the virtual sound source The sense of space on the horizontal plane also supplements and corrects the phase of the virtual sound source with amplitude translation according to the current head contour parameters, so as to reconstruct the sense of space of the virtual sound source in the vertical direction. Therefore, the audio output by the sounding device of the present application The signal can enable the wearer to better perceive the real orientation of the sound source, avoid the localization effect of the sound source, and improve the spatial sense of the sound field and user experience of the headset.

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: calculate the head-related transfer function based on the current head contour parameters of the wearer of the head-mounted device, and use the head-related transfer function as Phase modulation parameters.

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: determine the sound emitting device to be worked among all the sound emitting devices based on the position of the virtual sound source relative to the head-mounted device; control the head The sounding device of the wearable device performs stereo modulation on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and outputs the modulated audio signal. The parameter performs stereo modulation on the original audio signal and outputs the modulated audio signal.

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: obtain the incident direction of the virtual sound source based on the position of the virtual sound source relative to the head-mounted device; In the interval, determine the target sound image control interval where the incident direction is located, and reconstruct a sound image control interval for every two adjacent sound emitting devices; select the sound emitting device to be operated among the sound emitting devices in the reconstructed target sound image control interval.

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: obtain the projection of the incident direction on the horizontal plane; determine the sound image control interval where the projection is located as the target sound image control interval .

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps can be implemented: determine whether the projection is on any connection line dividing the sound image control interval; if so, reconstruct the target sound image Among the two sounding devices in the control section, the sounding device corresponding to the line where the projection is located is taken as the waiting sounding device; if not, the two sounding devices in the control section of the reconstructed target sound image are both taken as the waiting sounding device.

As an optional embodiment, when the processor executes the computer subroutine stored in the memory, the following steps may be implemented: determining and reconstructing the two sounding devices of the target sound image control interval based on the midpoint position of the human head The bisector of the included angle;

Calculate the respective amplitude modulation parameters of the two sounding devices according to the first relational expression, the first relational expression is

in,

is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.

On the basis of the above-mentioned embodiments, as a preferred implementation mode, refer to Figure X, which is a structural diagram of another head-mounted device provided by the embodiment of the present application. The head-mounted device also includes:

The input interface is connected with the processor, and is used to obtain the computer program, parameters and instructions imported from the outside, and store them in the memory under the control of the processor. The input interface can be connected with an input device to receive parameters or instructions manually input by the user. The input device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the terminal shell.

The display unit is connected with the processor and used for displaying the data sent by the processor. The display unit may be a liquid crystal display or an electronic ink display.

The network port is connected with the processor and is used for communication connection with various external terminal devices. The communication technology used in the communication connection can be wired communication technology or wireless communication technology, such as mobile high-definition link technology (MHL), universal serial bus (USB), high-definition multimedia interface (HDMI), wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, communication technology based on IEEE802.11s, etc.

On the other hand, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the audio output control method as described in any one of the above embodiments is implemented. step.

Specifically, the computer storage medium may include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc., which can store program codes. medium. A computer program is stored on the storage medium, and when the computer program is executed by the processor, the following steps are implemented: obtaining the initial audio signal of the virtual sound source in the current scene where the head-mounted device is located; based on the current human head contour parameters of the wearer of the head-mounted device Calculate the phase modulation parameters; calculate the amplitude modulation parameters based on the position of the virtual sound source relative to the head-mounted device; control the sounding device of the head-mounted device to modulate the stereo sound effect of the initial audio signal according to the amplitude modulation parameters and phase modulation parameters and output the modulated audio signal.

It can be seen that in this embodiment, the amplitude modulation parameters of the sounding device are calculated according to the position of the virtual sound source relative to the head-mounted device, and at the same time, the current head contour parameters of the wearer of the head-mounted device are obtained, and the phase modulation of the sounding device is calculated based on the head contour parameters parameter, and then control the sounding device to modulate the stereo sound effect on the initial audio signal of the virtual sound source according to the amplitude modulation parameter and the phase modulation parameter, and then output it. This application not only considers the magnitude of the sound output by the sounding device, but also reconstructs the virtual sound source The sense of space on the horizontal plane also supplements and corrects the phase of the virtual sound source with amplitude translation according to the current head contour parameters, so as to reconstruct the sense of space of the virtual sound source in the vertical direction. Therefore, the audio output by the sounding device of the present application The signal can enable the wearer to better perceive the real orientation of the sound source, avoid the localization effect of the sound source, and improve the spatial sense of the sound field and user experience of the headset.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: calculating the head-related transfer function based on the current head profile parameters of the wearer of the head-mounted device, and using the head-related transfer function as a phase modulation parameter.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: based on the position of the virtual sound source relative to the head-mounted device, among all the sound-emitting devices, the sound-emitting devices to be worked are determined; the sound-emitting device of the head-mounted device is controlled to press The amplitude modulation parameter and phase modulation parameter modulate the initial audio signal for stereo sound effects and output the modulated audio signal. Modulate the stereo effect and output the modulated audio signal.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: obtaining the incident direction of the virtual sound source based on the position of the virtual sound source relative to the head-mounted device; determining where the incident direction is located in all sound image control intervals The target sound image control interval of each two adjacent sound emitting devices reconstructs a sound image control interval; select the sound emitting device to be operated among the sound emitting devices of the reconstructed target sound image control interval.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: obtaining the projection of the incident direction on the horizontal plane; determining the sound image control interval where the projection is located as the target sound image control interval.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: judging whether the projection is on any connection line dividing the sound image control interval; if so, reconstructing the two vocalizations of the target sound image control interval In the device, the sounding device corresponding to the line where the projection is located is regarded as the sounding device to be operated; if not, the two sounding devices in the control area of the reconstructed target sound image are both regarded as the sounding device to be operated.

As an optional embodiment, when the computer program is executed by the processor, the following steps are implemented: determining the bisector of the angle between the two to-be-operated sounding devices based on the midpoint position of the human head; determining the reconstruction based on the midpoint position of the human head The bisector of the angle between the two sounding devices in the target sound image control interval; the respective amplitude modulation parameters of the two sounding devices are calculated according to the first relational expression, and the first relational expression is

in,

is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.

It should also be noted that in this specification, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations There is no such actual relationship or order between the operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. An audio output control method, characterized in that, comprising:

   Obtain the initial audio signal of the virtual sound source in the current scene where the headset is located;

   calculating phase modulation parameters based on current head profile parameters of the wearer of the head-mounted device;

   calculating an amplitude modulation parameter based on a position of the virtual sound source relative to the head-mounted device;

   Controlling the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.
2. The audio output control method according to claim 1, wherein the phase modulation parameter includes a head-related transfer function.
3. The audio output control method according to claim 1, wherein the head-mounted device includes n sound emitting devices, and n is an integer greater than 2;

   The audio output control method also includes:

   determining a to-be-operated sound emitting device among all the sound emitting devices based on the position of the virtual sound source relative to the head-mounted device;

   The process of controlling the sounding device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal includes:

   Controlling the standby sound generating device in the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.
4. The audio output control method according to claim 3, wherein each two adjacent sound emitting devices reconstruct a sound image control interval;

   The process of determining the sound emitting device to be operated among all the sound emitting devices based on the position of the virtual sound source relative to the head-mounted device includes:

   obtaining an incident direction of the virtual sound source based on a position of the virtual sound source relative to the head-mounted device;

   determining the target sound image control interval where the incident direction is located in all the sound image control intervals;

   Selecting a to-be-worked sound emitting device among the sound emitting devices reconstructing the target sound image control interval.
5. The audio output control method according to claim 4, wherein the n sound emitting devices are located on the same horizontal plane, and the line connecting the n sound emitting devices and the midpoint of the human head on the horizontal plane divides the horizontal plane being n said sound and image control intervals;

   The process of determining the target sound image control interval where the incident direction is located in all the sound image control intervals includes:

   Obtain the projection of the incident direction on the horizontal plane;

   The sound image control interval where the projection is located is determined as a target sound image control interval.
6. The audio output control method according to claim 5, wherein the process of selecting a sound emitting device to be operated from among the sound emitting devices reconstructing the target sound image control interval comprises:

   judging whether the projection is on any of the lines dividing the audio-image control interval;

   If yes, among the two sound emitting devices reconstructing the target sound image control interval, the sound emitting device corresponding to the line where the projection is located is used as the sound emitting device to be operated;

   If not, the two sound emitting devices reconstructing the target sound image control interval are both used as the waiting sound emitting devices.
7. The audio output control method according to any one of claims 1-6, wherein the process of calculating amplitude modulation parameters based on the position of the virtual sound source relative to the head-mounted device comprises:

   determining and reconstructing the bisector of the angle between the two sounding devices in the target sound image control interval based on the position of the midpoint of the human head;

   Calculate the respective amplitude modulation parameters of the two sounding devices according to the first relational expression, the first relational expression is

   

   in,

   

   is the angle between any one of the sounding device and the midpoint of the human head and the bisector,

   

   is the angle between the projection and the bisector, g1 is the amplitude modulation parameter of the first sounding device, and g2 is the amplitude modulation parameter of the second sounding device.
8. An audio output control system, characterized in that it comprises:

   An acquisition module, configured to acquire an initial audio signal of a virtual sound source in the current scene where the head-mounted device is located;

   A first calculation module, configured to calculate a phase modulation parameter based on the current head profile parameters of the wearer of the head-mounted device;

   A second calculation module, configured to calculate an amplitude modulation parameter based on the position of the virtual sound source relative to the head-mounted device;

   The output control module is used to control the sound generating device of the head-mounted device to modulate the stereo sound effect on the initial audio signal according to the amplitude modulation parameter and the phase modulation parameter and output the modulated audio signal.
9. A head-mounted device, characterized in that it includes:

   memory for storing computer programs;

   A processor, configured to implement the steps of the audio output control method according to any one of claims 1-7 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the audio output control according to any one of claims 1-7 is realized method steps.

Images