TW202241148A - Audio signal processing method and audio signal processing apparatus - Google Patents

Abstract

An audio signal processing method is provided. The method separates a left and a right channel audio signal to a center-left and center-right channel audio signals, and a side-left and side-right channel audio signals, and processes the center and side channel audio signals by a Head Related Transfer Functions (HRTF) so as to simulate the audio signals into several audio source positions related to a user, then synthesizes the processed audio signals into dual channel audio signals according to the user's hearing characteristics by a frequency compensation process. The invention further provides an audio signal processing apparatus to execute the audio signal processing method.

Description

Audio processing method and audio processing device

The invention relates to an audio processing technology, in particular to an audio processing method and an audio processing device for simulating two-channel audio into multi-channel audio.

Nowadays, multimedia information is increasingly developed, whether it is movies, TV dramas or games, etc. are inseparable from people's lives. Because of this, people often wear headphones to listen to music, watch movies or TV shows when commuting, and also wear headphones to play 3D games for a more immersive sound experience. However, the general traditional binaural earphones can only give the user two-channel listening experience, so that when watching movies or TV shows, the sound heard by the user will not have a sense of surround, or in When playing games, it is also impossible to distinguish sounds from multiple directions. In addition, each user's hearing performance is different, and different users will have different responses to different sound frequencies. Therefore, if there is an earphone that can process a two-channel sound source into multiple channels, and can adjust the frequency of the output sound according to the hearing characteristics of different users, it will definitely bring users a better listening experience.

The present invention provides an audio processing method, which can process a two-channel audio source into multi-channel, and can compensate audio of different frequencies in response to different users

The present invention further provides an audio processing device capable of executing the above-mentioned audio processing method

The present invention proposes an audio processing method, including: separating left channel audio into center left channel audio and side left channel audio; separating right channel audio into center right channel audio and side right channel audio; subjecting the center left channel audio and the center right channel audio to the center head transfer function, so as to simulate the center left channel audio and the center right channel audio as the first sound source position and the second sound source position relative to the user; subjecting the side left channel audio and the side right channel audio to the side head transfer function to model the side left channel audio and the side right channel audio as a third source position relative to the user and The fourth sound source position; and performing frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the user's hearing characteristics to synthesize binaural audio.

The present invention further proposes an audio processing device, which includes a channel separation unit, an audio computing unit, and an audio synthesis unit. The channel separation unit receives left channel audio and right channel audio, separates left channel audio into center left channel audio and side left channel audio, and separates right channel audio into center right channel audio and side Right channel audio. The audio computing unit processes the center left channel audio and the center right channel audio through the center head transfer function, so as to simulate the center left channel audio and the center right channel audio as relative to the user's first sound source position and second The position of the sound source, and the side left channel audio and the side right channel audio are processed by the side head transfer function, so that the side left channel audio and the side right channel audio are simulated as the first relative to the user Three sound source positions and a fourth sound source position. The audio synthesis unit performs frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the user's hearing characteristics to synthesize binaural audio.

In some embodiments, the audio processing method includes playing multiple tones with different frequencies to the user, and generating multiple frequency response values in response to the tones with different frequencies to obtain the hearing characteristics of the user, and then converting the frequency response values Compared with the preset value, at least one frequency response difference is generated, and then audio signals of different frequencies are compensated according to the frequency response difference.

In some embodiments, the audio processing device further includes an audio playback unit and a comparison unit. The audio playing unit plays a plurality of audios with different frequencies to the user, and the aforementioned audio computing unit generates a plurality of frequency response values in response to the audios with different frequencies to obtain the hearing characteristics of the user. The comparison unit compares these frequency response values with at least one preset value to generate at least one frequency response difference, and the aforementioned audio computing unit compensates the audio with different frequencies according to the frequency response difference.

To sum up, the audio processing method and the audio processing device provided by the present invention can process the audio from the left and right channels into audio from four different source directions, and can compensate the audio of different frequencies according to the hearing performance of the user, so that Binaural audio can also provide a surround sound listening experience.

The features, purpose and functions of the present invention will be further disclosed below. However, what is described below is only an embodiment of the present invention, and should not be used to limit the scope of the present invention, that is, all equivalent changes and modifications made according to the patent scope of the present invention will still be the gist of the present invention. It does not depart from the spirit and scope of the present invention, so it should be regarded as a further embodiment of the present invention.

FIG. 1 is a flowchart illustrating an audio processing method according to an embodiment of the present invention. As shown in FIG. 1 , the audio processing method of the present invention includes steps 101-105. Step 101: Separate the left channel audio into center left channel audio and side left channel audio. Step 102: Separate the right channel audio into center right channel audio and side right channel audio. Step 103: Process the center left channel audio and the center right channel audio with the center head transfer function, so as to simulate the center left channel audio and the center right channel audio as relative to the user's first sound source position and second The location of the sound source. Step 104: Perform side head transfer function processing on the side left channel audio and the side right channel audio, so as to simulate the side left channel audio and the side right channel audio as the third relative to the user The position of the sound source and the position of the fourth sound source. Step: 105: and performing frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the user's hearing characteristics to synthesize binaural audio.

FIG. 2 is a block diagram of an audio processing device according to an embodiment of the present invention. FIG. 3 is a schematic diagram illustrating the distribution of audio sources according to an embodiment of the present invention. Hereinafter, the blocks in FIG. 2 will be used to illustrate how the audio processing device of the present invention executes the audio processing method in FIG. 1 . Please refer to Figure 1, Figure 2 and Figure 3. As shown in FIG. 2 , the audio processing device 200 includes a stereo separation unit 201 , an equalizer 202 , a channel separation unit 203 , an audio computing unit 204 and an audio synthesis unit 205 . The stereo splitting unit 201 receives the stereo audio SA and splits it into left channel audio L and right channel audio R. In this embodiment, the stereo audio SA includes, for example, the left channel audio L and the right channel audio R, but the present invention is not limited thereto, and it may also include more channels. The equalizer 202 receives the left-channel audio L and the right-channel audio R, and the left-channel audio L and the right-channel audio R processed by the equalizer 202 can enhance their bass effects, thereby producing left-channel audio with better bass effects. channel audio L_Eq and right channel audio R_Eq. The channel separation unit 203 receives the left channel audio L_Eq and the right channel audio R_Eq, and separates the left channel audio L_Eq into the center left channel audio Cent_L and the side left channel audio Side_L, and separates the right channel audio R_Eq It is the center right channel audio Cent_R and the side right channel audio Side_R.

Specifically, the center left channel audio Cent_L and the center right channel audio Cent_R are equivalent to the left and right sound sources from the front for the user, while the side left channel audio Side_L and the side right channel audio Side_R are Equivalent to left and right sound sources from the sides. In particular, the above-mentioned left channel sound source and right channel sound source are the audio heard by the user's left ear and right ear respectively. The audio computing unit 204 receives the above-mentioned center left channel audio Cent_L, center right channel audio Cent_R, side left channel audio Side_L, and side right channel audio Side_R, and performs center head transfer function and side Edge head transfer function processing. Head Related Transfer Functions (HRTF) is a sound localization algorithm. The process of its localization and calculation is known to those skilled in the art, so it will not be repeated here. Through the operation of the head transfer function, the center left channel audio Cent_L, the center right channel audio Cent_R, the side left channel audio Side_L, and the side right channel audio Side_R will be virtualized as the sound source position 301 relative to the user 300 -304, as shown in Figure 3. The audio synthesizing unit 205 will receive the aforementioned audio Cent_LH, Cent_RH, Side_LH, Side_RH processed by the head transfer function, and then perform frequency compensation on the received audio according to the user's hearing characteristics to synthesize a two-channel audio. Now, even through binaural headphones, users can listen to the equivalent of multi-channel surround sound.

Furthermore, the audio processing device 200 is, for example, a two-channel earphone, and the stereo separation unit 201, the equalizer 202, the channel separation unit 203, the audio computing unit 204, and the audio synthesis unit 205 are independent or integrated in the earphone. components, circuits or chips. The audio processing device 200 further includes an audio playback unit and a comparison unit (not shown in the figure), the audio playback unit will play a plurality of audios with different frequencies to the user, and the user will give feedback in response to these audios with different frequencies after listening. This produces a number of frequency response values that can represent the hearing personality of the user. The comparing unit then compares these frequency response values with a preset value to generate at least one frequency response difference. The computing unit 204 can compensate the corresponding audio according to the frequency response difference. In this way, the user can not only have the experience of surround sound effect when listening, but also can obtain hearing compensation for the frequency audio with poor identification ability, further optimizing listening feelings. In particular, there are many methods of compensation. In this embodiment, for example, the compensation is a preset value, but the present invention is not limited thereto. Those skilled in the art can also compensate according to the design or the hearing characteristics of the user. for other values.

More specifically, the audio computing unit 204 performs filtering processing on the received center left channel audio Cent_L, center right channel audio Cent_R, side left channel audio Side_L, and side right channel audio Side_R. In the embodiment, for example, high-pass filtering is used to filter out bands that are relatively insensitive to human hearing sensitivity and retain bands that are more sensitive to human hearing sensitivity, but the present invention is not limited thereto. Those skilled in the art can also perform filtering in different bands according to design or user characteristics. The audio computing unit 204 performs a convolution operation on the filtered audio and a preset head transfer function. The convolution operation is known to those skilled in the art, so details are not repeated here. After the convolution operation, the audio can be virtualized to various directions relative to the user, thereby generating a listening experience of surround sound. In particular, the audio computing unit 204 can also multiply the convolved audio by a preset parameter. The preset parameter can be any value, which can optimize the intensity of the audio according to requirements.

In addition, the audio synthesis unit 205 can also receive the center left channel audio Cent_L and the center right channel audio Cent_R without the head transfer function processing, and combine the received audio without the head function transfer processing with the head transfer function The audio processed by the function is synthesized together, that is, the band that is less sensitive to human hearing sensitivity is synthesized with the band that is more sensitive. In this way, the synthesized audio can cover a wider band, thereby increasing the final The richness of the output audio.

To sum up, the audio processing method and audio processing device provided by the present invention simulate the original left and right channel audio into multi-channel audio through audio separation and head transfer function calculation, and can further adjust the simulated The audio is relative to the source direction of the user, and can further compensate the corresponding audio frequency for the user's hearing characteristics, so that the user can experience the optimized feeling of surround sound effect and hearing compensation.

101-105: Steps 200: Audio processing device 201: Stereo separation unit 202: equalizer 203: channel separation unit 204: Audio computing unit 205: Audio synthesis unit 300: user 301-304: Sound source location SA: stereo audio L: left channel audio R: right channel audio L_Eq: left channel audio with bass enhancement R_Eq: Right channel audio with bass enhancement Cent_L: Center left channel audio Cent_R: Center right channel audio Side_L: Side left channel audio Side_R: Side right channel audio Cent_LH: Center left channel audio after head transfer function processing Cent_RH: Center right channel audio after head transfer function processing Side_LH: Side left channel audio after head transfer function processing Side_RH: Side right channel audio after head transfer function processing

FIG. 1 is a flowchart illustrating an audio processing method according to an embodiment of the present invention.

FIG. 2 is a block diagram of an audio processing device according to an embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating the distribution of audio sources according to an embodiment of the present invention.

none

104~105: Steps

Claims (10)

An audio processing method, comprising: Split a left channel audio into a center left channel audio and a side left channel audio; Splitting a right channel audio into a center right channel audio and side right channel audio; subjecting the center left channel audio and the center right channel audio to a center head transfer function to simulate the center left channel audio and the center right channel audio as relative to a first audio source of a user position and a second sound source position; subjecting the side left channel audio and the side right channel audio to a side head transfer function to simulate the side left channel audio and the side right channel audio as relative to the user a third sound source position and a fourth sound source position; and The audio processed by the central head function transfer and the side head transfer function is subjected to frequency compensation according to the hearing characteristic of the user to synthesize a binaural audio. According to the audio processing method of claim 1, the step of performing the frequency compensation according to the hearing characteristic of the user includes: Play multiple tones with different frequencies to the user; Generate multiple frequency response values in response to the audio with different frequencies to obtain the hearing characteristics of the user; comparing the frequency response values with at least one preset value to generate at least one frequency response difference; and The audio signals with different frequencies are compensated according to the at least one frequency response difference. According to the audio processing method of item 2 of the scope of the patent application, the step of compensating the audio with different frequencies is compensating the frequency response values corresponding to the audio with different frequencies to the preset value. According to the audio processing method of claim 1, the audio processed by the central head transfer function and the side head transfer function is passed through a filter and a convolution operation. According to the audio processing method of item 4 of the scope of the patent application, the audio after the filtering and the convolution operation is further multiplied by a preset parameter. An audio processing device, comprising: A channel separation unit, receiving a left channel audio and a right channel audio, separating the left channel audio into a central left channel audio and a side left channel audio, and separating the right channel audio is a center right channel audio and a side right channel audio; An audio computing unit, receiving and performing a center head transfer function processing on the center left channel audio and the center right channel audio, so as to simulate the center left channel audio and the center right channel audio relative to a A first sound source position and a second sound source position of the user, and receive and process the side left channel audio and the side right channel audio with a side head transfer function, so that the side left channel audio and the side right channel audio are modeled as a third source position and a fourth source position relative to the user; and An audio synthesis unit receives and performs frequency compensation on the audio processed by the central head function transfer and the side head transfer function according to the hearing characteristics of the user to synthesize a binaural audio. The audio processing device according to item 6 of the scope of the patent application, wherein the audio computing unit has a filter, the center left channel audio, the center right channel audio, the side left channel audio and the side right sound A filtering process is performed on the channel audio. In the audio processing device according to claim 7 of the scope of the patent application, the audio computing unit performs a convolution operation on the filtered center left channel audio and the center right channel audio with the center head transfer function. The audio processing device according to item 7 of the scope of the patent application, wherein the audio computing unit performs a convolution on the filtered side left channel audio and the side right channel audio with the side head transfer function operation. The audio processing device according to item 7 of the scope of the patent application further includes: An audio playback unit, which plays a plurality of audios with different frequencies to the user, and the audio computing unit generates a plurality of frequency response values in response to the audios with different frequencies to obtain the hearing characteristics of the user; and A comparison unit compares the frequency response values with at least one preset value to generate at least one frequency response difference, and the audio computing unit compensates the audio signals with different frequencies according to the at least one frequency response difference.

Images