TW201843675A - Apparatus and method for downmixing multichannel audio signals - Google Patents

Abstract

A method for processing a multi-channel input audio signal is performed at a computing device. The method includes the following steps: selecting, from the multi-channel input audio signal, a left input channel and a right input channel, wherein the left input channel and the right input channel correspond to a pair of spatially symmetrical signal sources; generating one or more cross-channel features from the left input channel and the right input channel; processing, in accordance with the cross-channel features, the left input channel and the right input channel to generate a left intermediate channel and a right intermediate channel; and combining each of the left intermediate channel and the right intermediate channel with a third input channel of the multi-channel input audio signal to form a two-channel output audio signal.

Description

   Device and method for downmixing multi-channel audio signals   

The invention relates to an audio signal processing method, in particular to a computer implementation method and device for downmixing multi-channel audio signals and computer usable program code.

Surround sound is a technique that uses multiple channels around the listener to generate, transmit, and play audio, usually using multiple discrete channels. The two most common configuration methods for multi-channel or surround sound are 5.1 surround sound and 7.1 surround sound. A set of 5.1 surround sound configuration consists of a pair of front speakers (L and R), a center front channel (C), a pair of side speakers (Ls and Rs) and a low frequency effect channel (LFE), The traditional configuration sequence is L, C, R, Ls, Rs, LFE. A set of 7.1 surround sound configuration consists of a pair of front speakers (L and R), a center front channel (C), a pair of side surround speakers (Lss and Rss), and a pair of rear surround speakers (Lrs and Rrs) and a low-frequency effect channel (LFE), the traditional configuration sequence is L, C, R, Lss, Rss, Lrs, Rrs, LFE.

Downmixing is a method of converting audio content with a multi-channel configuration (for example, a multi-channel audio file) into less-channel audio content. For example, a 5.1 surround sound file or 7.1 surround sound file can be played back using a two-channel stereo playback system through downmix processing, and the dual-channel stereo playback system can still provide a good listening experience for listeners .

In the traditional downmix method, each multichannel audio input is processed independently and separately by individual processors to produce one or two channel outputs. The processing procedures on each channel do not properly consider the meaningful information between the two pairs of speakers. Therefore, the audio output obtained by using such traditional downmix methods is relatively distorted, and may detract from the immersive listening experience.

The purpose of the present invention is to develop an audio downmix processing pipeline that processes input channels in pairs, which can make the output generated after downmixing have higher accuracy, while retaining the spatial information of the original multi-channel audio stream. Although the number of channel inputs and outputs can be any significant number within the defined range, the following description uses 5.1 channel downmixing to stereo and 7.1 channel downmixing to stereo as examples.

One aspect of the present invention provides a multi-channel input audio signal processing method executed on a computing device. The computing device has one or more processors, a memory, and the one or more processors stored in the memory Multiple program modules executed. The method includes the following steps: selecting a left input channel and a right input channel from the multi-channel input audio signal, wherein the left input channel and the right input channel correspond to a pair of spatially symmetric signal sources Generating one or more cross-channel features from the left input channel and the right input channel; processing the left input channel and the right input channel according to the cross-channel features to generate a left center channel And a right center channel; and the left center channel and the right center channel, respectively, are combined with a third input channel of the multi-channel input audio signal to form a two-channel output audio signal.

Another aspect of the present invention provides a computing device, including: one or more processors; a memory; and a plurality of program modules stored in the memory and executed by the one or more processors. When the plurality of program modules are executed by one or more processors, the computing device can enable the above-mentioned method of processing multi-channel input audio signals. Another aspect of the present invention provides a computer program product, which is stored in a non-transitory computer readable storage medium, the storage medium is connected to a computing device, and the computing device has one or more processors, the computer program product It includes a plurality of program modules. When the program modules are executed by one or more processors, the computing device will execute the above method for processing multi-channel audio signals.

100‧‧‧Data processing system

102‧‧‧Communication architecture

104‧‧‧ processor unit

106‧‧‧Memory

108‧‧‧Persistent storage

110‧‧‧ communication unit

112‧‧‧I / O unit

114‧‧‧Monitor

116‧‧‧speaker

118‧‧‧ Computer can read the storage media

120‧‧‧Code / module

122‧‧‧Computer program products

222, 224‧‧‧ signal pair

232, 234‧‧‧ PROC

240‧‧‧7.1 surround sound file

242, 244, 246‧‧‧ signal pair

252, 254, 254, 256‧‧‧ PROC

410‧‧‧ Input signal pair

420‧‧‧PROC / signal pair processor

422‧‧‧M / S mixer

424‧‧‧Side signal

426‧‧‧Central component (M)

428‧‧‧Equalizer (EQ)

430‧‧‧Dynamic Range Compressor (DRC)

432‧‧‧crosstalk cancellation module (XTC)

434‧‧‧Width Controller (WC)

436‧‧‧Dynamic Range Compressor (DRC)

440‧‧‧ Output signal pair

450‧‧‧band-pass filtered signal

452‧‧‧ Residual signal

462, 464, 466

472, 474‧‧‧ output signal pair

476‧‧‧Output signal

482, 484, 486‧‧‧ width controller (WC)

488‧‧‧stereo output signal

500‧‧‧User Interface (UI)

510‧‧‧Left panel

520, 530, 540‧‧‧ control area

602 to 630‧‧‧ steps

The accompanying drawings in this specification provide a further understanding of specific embodiments of the present invention, and constitute a part of the description, which depicts the specific embodiments and explain in detail the working principle of the present invention together with the description. Similar component numbers are used to indicate similar corresponding components.

The first diagram A is a block diagram illustrating the implementation of a conventional downmix method on a 5.1-channel input signal according to some specific embodiments.

Figure B is a block diagram illustrating the implementation of a conventional LoRo downmix method on a 5.1-channel input signal according to some specific embodiments.

The first C diagram is a block diagram illustrating a conventional downmix method for performing surround sound virtualization or spatialization on a 7.1-channel input signal according to some specific embodiments.

The second diagram is a block diagram illustrating a data processing system configured to perform audio downmixing according to an exemplary embodiment of the present invention.

Figures 3A to 3B are block diagrams illustrating audio downmix pipelines for processing multi-channel input signals according to some specific embodiments.

The fourth diagram A is a block diagram of a signal processing flow according to some specific embodiments. The signal processing flow includes a PROC applied to an input signal pair.

Figure 4B is a block diagram illustrating a signal processing flow applied to a 7.1 surround sound file according to some specific embodiments.

The fifth diagram illustrates the user interface of a software application or its plug-in components according to some specific embodiments, which is used to manage the execution of the 7.1 surround sound file signal processing pipeline shown in the fourth diagram.

Figures 6A to 6C are flowcharts illustrating a method for downmixing a multi-channel audio signal according to some specific embodiments.

The following is a detailed description of specific embodiments of the present invention, examples of which are depicted in the accompanying drawings. The following description will reveal several non-limiting specific features in detail to assist in understanding the subject matter of the present invention. Those of ordinary skill in the art can clearly know that the present invention can be implemented in various alternative ways without departing from the scope of the present invention, and the subject matter of the present invention can be implemented without the specific details detailed here. For example, those of ordinary skill in the art will clearly know that the subject of the present disclosure can be implemented in a variety of radio communication systems, such as smart phones and tablet computers.

Please refer to the drawings to understand the following description. The data processing environment illustrated in the block diagram in the drawings only illustrates the environment on which the exemplary embodiments can be implemented. It should be understood that the drawings referred to are merely exemplary in nature, and are not intended to claim or imply limitations on the environment that can be imposed on different embodiments, and the described environment still allows many modifications.

Please refer to FIG. 1A, which illustrates a block diagram of a conventional downmix method applied to a 5.1 input signal. As shown in Figure A, each input channel (ie: L, C, R, Ls, Rs, LFE), regardless of its relationship with other channels, is processed separately and sent to its respective processor module ( PROC). The processor may include one or more sub-modules (not shown), such as gain, time delay, low-pass filter, and / or other audio processing modules. The output of each processing module processing each channel may include one or more channels, depending on the type of program executed in the processing module. Finally, in this example, the sum of these outputs (ie: Σ) is a two-channel audio (ie: L and R output channels).

Please refer to FIG. 1B, which illustrates a block diagram of a conventional mono-left channel / mono-right channel (LoRo) downmix method applied to a 5.1 input signal. Each input channel (ie L, C, R, Ls, Rs, LFE) will pass through a gain module. The gain adjustment depends on the actual position as reproduced by the surround sound system. Although the surround channels may be weakened more than the left / right channels, the relationship between the left and right channels is ignored. The left channel output is generated by adding all the channels on the left and adding the weakened C and LFE signals. The center channel is divided into two channels, because in a stereo reproduction environment, there is no physical speaker on the center line. LFE is also divided into two channels. The right channel output is generated by adding all the channels on the right and adding the weakened C and LFE signals. In this example, each input channel is processed separately by a separate processor, which includes a simple gain module. The processor receives a mono input and generates a mono or dual output. Finally, all PROC outputs are summed up according to the reproduction position predetermined by the input.

Referring to FIG. 1C, this block diagram illustrates a conventional downmixing method for virtualizing or spatializing surround sound effects on a 7.1 input signal. In this example, for each channel of the input multi-channel audio (ie: L, C, R, Lss, Rss, Lrs, Rrs, LFE), in addition to the individual gain processing, it will also be represented by each entity The head-related transfer function (HRTF) at the predetermined position of the speaker is processed separately to generate a two-channel output. For example, the left channel input will be processed by the HRTF representing the left channel of the speaker in the surround sound system. Similar processing procedures are also applied to all other input channels. All the two-channel output groups generated according to the respective input channels will be added together to form the left channel output and the right channel output. In this example, each input channel is also processed separately by a separate processor (including, for example, a gain module and an HRTF filter). The processor will receive a mono input and generate a two-channel output. All two-channel outputs will be added together to form the final two-channel output.

The second diagram is a block diagram illustrating a data processing system 100 configured to perform audio downmix processing according to an exemplary embodiment of the present invention. In this example, the data processing system 100 includes a communication architecture 102 that provides a processor unit 104, a memory 106, a persistent storage 108, a communication unit 110, and an input / output (I / O) unit 112. Communication between a display 114 and one or more speakers 116. It should be noted that the speaker 116 may be built in the data processing system 100 or externally connected to the data processing system 100. In some embodiments, the data processing system 100 may be in the form of a notebook computer, a desktop computer, a tablet computer, a mobile phone (for example: a smartphone), a multimedia playback device, a navigation device, and an educational device (for example: a child Learning toys), game console systems, audio-visual (AV) receivers, or control devices (such as home or industrial controllers).

The function of the processor unit 104 is to execute software program instructions that can be loaded into the memory 106. The processor unit 104 may be a set including one or more processors, or may be a multi-core processor, depending on the specific tasks to be performed. Furthermore, the processor unit 104 may be implemented using one or more heterogeneous processor systems, where a primary processor and a secondary processor are co-located on a single chip. In another example, the processor unit 104 may be a symmetric multi-processor system, including multiple processors of the same type.

In these examples, the memory 106 may be random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 108 can take different forms depending on the specific job execution. For example, persistent storage 108 may include one or more components or devices, such as a hard disk, flash memory, rewritable optical disc, rewritable magnetic tape, or a combination of the above components or devices. The media used by persistent storage 108 may also be removable. For example, persistent storage 108 may use removable hard drives.

In these examples, the communication unit 110 provides communication with other data processing systems or devices. In these examples, the communication unit 110 is a network interface card. The communication unit 110 can provide a communication function by using one or both of physical and wireless communication links.

The input / output unit 112 provides data input or output with other devices that can be connected to the data processing system 100. For example, the input / output unit 112 can provide a user with an input connection device through a keyboard and a mouse. Moreover, the input / output unit 112 can send and output to a printer. The display 114 provides an information display mechanism to the user. The speaker 116 plays sound effects to the user.

Operating system commands and applications or programs are located on persistent storage 108. These instructions may be loaded into the memory 106 for execution by the processor unit 104. The processing procedures described in the following embodiments can be executed by the processor unit 104 using computer-implemented instructions located on a memory (for example, the memory 106). These instructions are called program code (or module), the computer can use the program code (or module), or the computer can read the program code (or module), which can be read by a processor in the processor unit 104 and carried out. The code (or modules) in these different embodiments may be embodied on various types of physical or tangible computer-readable media, such as memory 106 or persistent storage 108.

The code / module 120 is located on a functional form of a computer-readable storage medium 118. The computer-readable storage medium 118 is optionally removable, and the code / module 120 can also be loaded or transferred to The data processing system 100 is executed by the processor unit 104. In these examples, the program code / module 120 and the computer-readable storage medium 118 constitute a computer program product 122. In one example, the computer-readable storage medium 118 may be in a tangible form, such as an optical disc or disk inserted or inserted into a hard drive or other device in a portion of persistent storage 108 to transfer data A storage device, such as a hard drive that is part of persistent storage 108. In a tangible form, the computer-readable storage medium 118 may also be a form of persistent storage connected to the data processing system 100, such as a hard disk, pen drive, or flash memory. The tangible form of the computer-readable storage medium 118 may also be referred to as computer-recordable storage medium. In some examples, the computer-readable storage medium 118 may not be removable from the data processing system 100.

Alternatively, the program code / module 120 may be transmitted from the computer-readable storage medium 118 to the data processing system 100 through a communication link with the communication unit 110 and / or through a component connected to the input / output unit 112. In an example, this communication link and / or connection member may be physical or wireless. The computer-readable media can also be in the form of non-tangible media, such as: communication link method or wireless transmission method including code / module.

The different components shown in the data processing system 100 are not intended to provide architectural limitations to the possible implementation forms of different embodiments. These different exemplary embodiments may be implemented on a data processing system, which may include components other than those shown in the data processing system 100, or include other alternative components. The other components shown in the first figure may be different from the components shown in the examples.

In one example, a storage device of the data processing system 100 is any hardware device that can store data. Memory 106, persistent storage 108, and computer-readable storage medium 118 are all examples of tangible storage devices.

In another example, a bus system can be used to implement the communication architecture 102 and consist of one or more bus bars, such as a system bus or input / output bus. The implementation of the bus system can use any suitable architecture type to provide data transmission between different components or devices connected to the bus system. In addition, the communication unit may include one or more devices for transmitting and receiving data, such as a modem or a network adapter. Furthermore, the memory here may be, for example, the memory 106 or the cache memory, which may exist in the interface and the memory controller hub in the communication architecture 102.

In order to solve the problems of the conventional methods described in the background of the present invention, the following describes different specific embodiments of the present invention, which are related to the system and method of paired downmix input channels to provide better accuracy of audio information, and Retain the spatial information of the original multi-channel audio stream. Different from the traditional method, the relationship between the input channels is considered in pairs, and each signal pair is separately transmitted to a processor. The signal information contained in a single signal pair will be compared and analyzed with each other to create a stronger audio-visual and better spatial effect. In order to make the signal pair meaningful, at least one module in the processing program needs to cross-reference the signal information contained in the signal to the signal itself. The dual channel output of each signal pair will be summed with the mono channel to produce a left channel output and a right channel output.

Figures 3A to 3B are block diagrams illustrating an audio downmix pipeline for processing multi-channel input signals according to some embodiments of the present invention. The multi-channel input signal in the third diagram A is a 5.1 surround sound file 210, which includes a left front channel (L), a center front channel (C), and a right front channel (R) , A left channel (Ls), a right channel (Rs) and a low frequency effect channel (LFE). The multi-channel input signal in Figure 3B is a 7.1 surround sound file 240, which includes a left front channel (L), a center front channel (C), and a right front channel (R) , A left surround channel (Lss), a right surround channel (Rss), a left surround channel (Lrs), a right surround channel (Rrs) and a low frequency effect channel (LFE) ).

In some embodiments, the system selects one or more input signal pairs from multi-channel input signals. In some embodiments, an input signal pair corresponds to two sets of audio streams to be reproduced on symmetrical two-sided speakers. Therefore, the input signal pair includes a pair of spatially symmetrical signal sources. In some embodiments, an input signal pair includes two channels, which are located on both sides (ie, left and right) and maintain the same angle with the center line. For example, the front pair includes the left front (L) channel and the right front (R) channel, and the left and right sides are each maintained at an angle of 30 ° to the center line. In another example, the Dolby 7.1 surround sound setting, the rear pair includes the left rear surround (Lrs) channel and the right rear surround (Rrs) channel, and the left and right sides maintain a 135 ° angle with the center line. Each selected input signal pair is then sent to a respective processor (PROC) to generate an output audio signal pair.

In some embodiments of the present invention, as shown in FIG. 3A, the system selects the left front channel (L) and the right front channel (R) from the multiple input channels of the 5.1 surround sound file as a signal Pair 222, and select the left channel (Ls) and the right channel (Rs) to become a signal pair 224. In some embodiments, as shown in Figure 3B, the system selects the left front channel (L) and the right front channel (R) from the multiple input channels of the 7.1 surround sound file to become a signal pair 242 , Select the left surround channel (Lss) and right surround channel (Rss) as a signal pair 244, and select the left rear surround channel (Lrs) and right rear surround channel (Rrs) as a signal pair 246. The channels located on the center line (for example: center front channel C) and omnidirectional channels (for example: LFE) are each a single channel, and will not be paired with any other channel.

In some embodiments, each signal pair will be transmitted into a processor. For example, in the third diagram A, the signal pairs 222 and 224 are transmitted to PROC 232 and 234, respectively; in the third diagram B, the signal pairs 242, 244, and 246 are transmitted to PROC 252, 254, and 256, respectively. In this procedure, the signal information contained in a single signal pair will be compared and analyzed with each other to create a stronger audio-visual and better spatial effect. To make the signal pair meaningful, among the one or more modules included in each processor PROC, at least one module needs to cross-reference the information between the two channels in a signal pair. The output signal of each processor PROC includes two channels, and the two channel output of each signal pair will be summed with the output of other single channels (Σ) to produce a left channel output (L ') and The output signal of a right channel output (R ') (as shown in Figures A and B respectively).

In some embodiments, the signal-to-processor (PROC) may be any possible "two-in, two-out" audio signal processor. As mentioned above, the processor includes at least one module that combines cross-channel features from an input signal pair. In some embodiments, a signal pair processor (PROC) includes one or more modules that read signal information from an input signal pair. Based on the signal information of this input signal pair, the signal pair processor (PROC) then changes the output stream on a paired basis.

In some embodiments, a signal pair processor (PROC) is composed of a plurality of different components (or modules), including channel independent processing components and / or channel independent processing components. In some embodiments, the channel-independent processing component executes a "multi-channel in, multi-channel out" processing program, for example, a signal to the two-in-two-out program in the processor. In some embodiments, the channel independent processing component generates multiple output channels, where each output channel is generated based on more than one input channel. In some embodiments, the channel independent processing component uses the information of the input signal and adjusts the processing program based on the extracted cross-channel features. In some embodiments, the cross-channel characteristics include the comparison of the volume of each input channel, the relationship between the spectral characteristics (eg, intensity and / or phase) of the left and right input channels, and / or the time difference between the start of the left and right input channel signals And amplitude difference. In some embodiments, the signal-to-processor (PROC) includes one or more channel-independent processing components, such as an M / S mixer (mid / side mixer) and / or Width controller (WC). For example, the M / S mixer can use the sum and difference of the left and right input signals to generate the center and side signals. In another example, by comparing the overlapping areas of the input signal spectrum, the M / S mixer can generate the center and both side signals.

In some embodiments, the channel independent processing component executes a "multi-channel in, multi-channel out" procedure (including two in and two out), and processes each individual channel of the multi-channel input file separately. In some embodiments, when the channel-independent processing component processes a multi-channel input file, multiple channels are treated as mono signals that are divided into the same number, and each mono signal is processed independently. Then, the mono signals after the processing of multiple channels are added up. In some embodiments, the signal pair processor (PROC) includes one or more channel independent processing components, such as an equalizer (EQ) and / or a dynamic range compressor (DRC). For example, an equalizer (EQ) module receives a single channel and generates the corresponding channel without using any information from other channels. The output of the equalizer (EQ) is that each channel is equalized using the same input parameters.

The fourth diagram A is a block diagram illustrating a signal flow of applying a PROC 420 to an input signal pair 410 according to some embodiments of the present invention. In some embodiments, PROC 420 includes a plurality of modules (also referred to as components) that are configured to process input signal pairs 410. In some embodiments, PROC 420 may be any signal pair processor, such as PROC 232, PROC 234, PROC 252, PROC 254, or PROC 256 shown in FIGS. 3A to 3B. Therefore, the signal pair processor 420 can be applied to any input signal pair, such as the signal pair 222, the signal pair 224, the signal pair 242, the signal pair 244, or the signal pair 246 shown in the third to third B diagrams. In some embodiments, the signal pair processor 420 includes an M / S mixer 422, an equalizer (EQ) 428, a dynamic range compressor (DRC) 430, and a crosstalk cancellation (XTC) module 432 The signal is coupled to the above components of the processor 420 as shown in the fourth diagram A. The output signal of PROC 420 is further processed by a width controller (WC) 434 and another dynamic range compressor (DRC) 436 to obtain an output signal pair 440, as shown in the fourth diagram A.

In some embodiments, the data processing system 100 first transmits the left and right input signals 410 into the M / S mixer 422. In some embodiments, the M / S mixer 422 is a mixing tool configured to generate three signal components (including two side components (S) 424 from the input signal pair 410, ie Left and right, and a central component (M) 426). The left component represents a source that only appears on the left channel, while the right component corresponds to a source that only appears on the right channel. The central component is the sound source of the phantom center that appears only in the sound field, for example: the main musical elements and dialogue.

In this way, the M / S mixer 422 can separate information that is helpful for various subsequent sound field enhancement processing, and minimize unnecessary distortion (eg, sound stain) in sound quality. Furthermore, this step also helps to reduce the correlation between the left and right components. In some embodiments, the M / S mixer 422 analyzes the audio image and estimates the sound effects from the center and left and right channels, and then the M / S mixer 422 converts the two input channels (ie, the left The right side signal 410) is divided into a mono-channel center signal 426 and a two-channel side signal 424. For a more detailed description of the M / S mixer 422, see the application for PCT named APPARATUS AND METHOD FOR SOUND STAGE ENHANCEMENT (Case No. PCT / US2015 / 057616), which was filed on October 27, 2015. The way of reference is included in this case.

Next, the system sends the side signal 424 to an equalizer (EQ) 428 to adjust the frequency component of the side signal 424. In some embodiments, the equalizer 428 into which the two side signals 424 enter includes one or more multi-band equalizers to perform a band-pass filtering function on the two side signals. In some embodiments, the multi-band equalizer applied to each side signal is the same. In some other embodiments, the multi-band equalizer applied to one side signal is different from the equalizer applied to the other side signal. However, its function is also to maintain the original timbre of the audio signal, and to avoid fuzzy spatial clues of the two signals. In some embodiments, the equalizer 428 can also be used to select the target sound source based on the spectrum analysis of the signal components on both sides. In some embodiments, as shown in the fourth diagram A, the equalizer 428 generates two output signals 450 and 452. In some embodiments, the output signals 450 and 452 are each two-channel audio signals. In some embodiments, the equalizer 428 uses a band-pass filter on each two-channel side signal 424 to obtain a band-pass filtered signal 450.

In some embodiments, the equalizer 428 also generates a residual signal 452 based on the band difference between its input signal (ie, the two-channel side signal 424) and the output signal (ie, the two-channel band-pass filtered signal 450). In some embodiments, the data processing system 100 generates a left residual component and a right residual by subtracting the left component and the right component from the left component after the band pass filtering process and the right component after the band pass filtering process, respectively Weight. In some embodiments, separate amplifiers are applied to the residual signal and the generated signal generated by the crosstalk cancellation process to adjust the gain before combining the two signals. For a more detailed description of the equalizer 428, see the application filed on October 27, 2015, named PCARATUS AND METHOD FOR SOUND STAGE ENHANCEMENT (Case No. PCT / US2015 / 057616), the full text of which is incorporated by reference Included in this case.

In some embodiments, the band-pass filtered signal 450 is transmitted to the dynamic range compressor (DRC) 430. In some embodiments, the dynamic range compressor 430 includes a band-pass filter (unlike the band-pass filter of the equalizer 428) for amplifying the two sound signals within a predetermined frequency range (ie: two sounds) The channel's band-pass filtered signal 450) to maximize the sound field enhancement effect achieved by the crosstalk cancellation module (XTC) 432. In some embodiments, a user (for example, an audio engineer) can adjust the band-pass filter of the dynamic range compressor 430 to make a specific frequency band stand out. In this way, the user can highlight some specific sound events of his choice. For example, the data processing system 100 uses the first bandpass filter of the equalizer 428, performs equalization processing on the left and right components, and then uses the second bandpass filter of the dynamic range compressor 430 for the left The component and the right component remove a predetermined frequency band. Representative bandpass filters that can be used by the equalizer module 428 and the dynamic range compressor module 430 include biquad filters or Butterworth filters. In some embodiments, the data processing system 100 uses the first band-pass filter of the equalizer 428 to perform equalization processing on the left and right components, and then uses the dynamic range compressor 430 to perform the first dynamic range on the left and right components Compressed to highlight predetermined frequency bands of other frequencies. For a more detailed description of the dynamic range compressor 430, see the application filed on October 27, 2015, named PCARATUS AND METHOD FOR SOUND STAGE ENHANCEMENT (Case No. PCT / US2015 / 057616), the full text of which is incorporated by reference Into the case.

In some embodiments, the signal is output from the dynamic range compressor 430 and then transmitted to the crosstalk cancellation (XTC) module 432 to perform the crosstalk cancellation process. Crosstalk is an existing problem in stereo (ie, two-channel) audio speakers. It will occur when the sound from each speaker reaches the opposite side of the ear, and it will bring unnecessary spectrum to the original signal. The solution to this problem is crosstalk cancellation (XTC) operation. One of the XTC operations is to use generalized directional binaural transfer functions, such as head-related transfer functions (HRTFs) and / or spatial binaural impulse response functions (BRIR), to represent the two physical speakers' relative to the listener's position. angle. Another XTC computing system is a recursive crosstalk cancellation method, which does not need to calculate the head transfer function (HRTF), spatial binaural impulse response function (BRIR), or other binaural transfer functions. The basic calculation method can be expressed by the following formula: left [ n ] = left [ n ] -A _L * right [ n - d _L ] right [ n ] = right [ n ] -A _R * left [ n - d _R ] Among them, A _L and A _R are the attenuation coefficients of the signal, and d _L and d _R are the number of data samples that are delayed from individual speakers to the opposite side ear. In some embodiments, the crosstalk cancellation module 432 shown in FIG. 4A uses a recursive crosstalk cancellation method or a generalized directional binaural transfer function. For a more detailed description of the crosstalk cancellation module 432, see the application filed on December 12, 2014, US Patent Application No. 14 / 569,490, named APPARATUS AND METHOD FOR SOUND STAGE ENHANCEMENT (Approved on December 27, 2016, Approved Patent No. 9,532,156), and an application filed on November 11, 2016, named US Patent No. 15 / 349,822, named APPARATUS AND METHOD FOR SOUND STAGE ENHANCEMENT. The full text of the above cases is incorporated by reference.

In some embodiments shown in FIG. 4A, the output signal of the crosstalk cancellation module 432 is sent to the amplifier 462, the residual signal pair 452 is sent to the amplifier 464, and the central component (M) 426 is also sent to The amplifier 466 then sends the above three signals to the width controller (WC) 434 for processing and combining.

In some embodiments, the output signals of the amplifiers 462, 464, 466 are sent to the width controller 434 to adjust the width of the sound field. In some embodiments, the width controller 434 uses the analyzed input signal pair information to control the width of the sound field of the output audio signal. The width of the sound field can range from the narrowest to 0 °, to 360 ° wide for fully immersive sound effects. In some embodiments, the cross-channel information of a signal pair (eg, output signal pair 472 or 474) is analyzed and adjusted. In some embodiments, the user can specify the desired width of the sound field on the width controller, as illustrated in the fifth figure below. The specified width of the sound field may be based on the previously analyzed information and affect the signal pair summation matrix of the width controller 434. In some embodiments, the width of the sound field can be adjusted by the following equation:

Where the sound field width parameter is -5 β 0. When β = 0, the generated signal has the maximum sound field width, and when β = -5, the generated signal is close to a mono signal.

In some embodiments, the output signal 476 of the width controller 434 is sent to the second dynamic range compressor (DRC) 436 to amplify the overall output level of the audio signal in the mastering post-production process. In some embodiments, the data processing system 100 uses the dynamic range compressor 436 to perform second dynamic range compression on the left component and the right component to save the positioning clues of the digital audio output signal.

As shown in the fourth diagram A, the output of this process pipeline is a stereo audio signal 440, which includes a left channel (L ') and a right channel (R'). In some embodiments, the signal processing flow including PROC 420 can be applied to the 5.1 surround sound file as shown in the third diagram A. For example, the PROC 232 and / or PROC 234 of the third diagram A may be similar to the PROC 420 illustrated in the fourth diagram A.

The block diagram of the fourth B diagram is a signal processing flow applied to a 7.1 surround sound file according to some specific embodiments of the present invention. Referring also to the third diagram B, in some embodiments, the input signals of the 7.1 surround sound file are classified into different signal pairs, namely, L / R signal pair 242, Lss / Rss signal pair 244, and Lrs / Rrs signal pair 246. Next, the L / R signal pair 242, the Lss / Rss signal pair 244, and the Lrs / Rrs signal pair 246 are transmitted to the PROC 252, PROC 254, and PROC 256, respectively. In some embodiments, each individual PROC 252, PROC 254, and PROC 256 are similar to the PROC 420 discussed in Figure 4A. In other embodiments, PROC 252, PROC 254, or PROC 256 may include one or more other modules (or components) in different configurations. The signals output by PROC 252, PROC 254, and PROC 256 are sent to their respective width controllers, such as width controller 482, width controller 484, and width controller 486, respectively. The width controller 482, the width controller 484, or the width controller 486 may be similar to the width controller 434 discussed in the fourth A figure. The output signals of the width controller 482, the width controller 484 and the width controller 486 are combined with the center signal C and the low-frequency effect channel signal LFE to generate a stereo audio output signal 488.

The fifth figure illustrates a user interface (UI) 500 of a software application or its plug-in components according to some embodiments of the present invention, which is used to manage the execution of the 7.1 surround sound file signal processing pipeline shown in the fourth figure B above. . The signal processing pipeline may include a plurality of signal pair processors, such as the PROC 420 illustrated in FIG. 4A. In this example, there are three sets of signal pairs: front signal pairs (L and R), side signal pairs (Lss and Rss), and rear signal pairs (Lrs and Rrs). The left panel 510 of the user interface 500 controls the gain of each input channel. The control area 520 controls the frequency component of the equalizer (EQ). The control area 530 controls the width of the width controller. As described with reference to FIG. 4B, each signal pair passes through a different signal pair processor PROC, and each PROC uses different parameters, so the control area 540 can be used to select a signal pair to be processed (eg: front Set signal pair, side signal pair or post signal pair) to input the parameters when executing the PROC processing program.

As mentioned above, sound plays an important role in media and entertainment, producing an emotional effect on the audience and connecting them to the story. To enable listeners to have a more immersive listening experience, they developed a multi-channel surround sound effect. The multi-channel audio format uses multiple audio tracks to reconstruct sound effects on a corresponding multi-channel sound reproduction system. Downmixing can be done on the audio generation and reproduction side. On the generation side, the mixer usually starts with the largest number of channels and then downmixes to a smaller number of channels. On the reproduction side, a multi-channel audio track can be downmixed to a smaller number of channels to match the number of channels in the reproduction system. In both cases, the goal is to maintain the use and configuration of sound effects consistent with the original creative concept.

The traditional downmixing method is shown in Figures A to C, and a processing flow is applied to individual audio tracks. The relationship between audio tracks is not considered. The method and system proposed in this paper consider audio input on a paired basis. First, the multi-channel audio tracks are classified into pairs according to the physical position relationship of the reproduction system. Second, analyze the relationship between the two channels in a signal pair. Finally, based on the analysis result, the signal pair input is processed. By incorporating the relationship between the signal pairs, the spatial information of the original multi-channel audio input can be preserved better. Therefore, for the same audio input, the multi-channel audio output generated by the downmix processing using the system and method described in this article will be closer to the original multi-channel audio input than the traditional method of downmixing, and create a more Accurate sound field.

FIGS. 6A to 6C are flowcharts illustrating the multi-channel audio signal downmixing using the data processing system 100 as shown in FIG. 2 according to some embodiments of the present invention. The data processing system 100 selects (step 602) a left input channel and a right input channel from the multi-channel audio input signal. In some embodiments, the left input channel and the right input channel correspond to a pair of spatially symmetric signal sources. In some embodiments, the multi-channel audio input signal is, for example, the 5.1 surround sound effect file 210 in the third A diagram, or the 7.1 surround sound effect file 240 in the third B diagram. In some embodiments, the 5.1 surround sound input signal 210 includes a left front channel L and a right front channel R, which form a signal pair 222, and a left channel Ls and a right channel Rs, It forms a signal pair 224. In some embodiments, the 7.1 surround sound input signal 240 includes a left front channel L and a right front channel R, which form a signal pair 242; a left surround channel Lss and a right surround channel Rss , Which forms a signal pair 244; and a left rear surround channel Lrs and a right rear surround channel Rrs, which form a signal pair 246.

The data processing system 100 then generates (step 604) one or more cross-channel features from the left and right input channels of a selected signal pair. In some embodiments, the one or more cross-channel features include comparing the volume of a signal to the left and right input channels, the relationship between the spectral characteristics (eg, intensity and / or phase) of the left and right input channels, and / or The time difference and amplitude difference of the left and right input channel signal start.

The data processing system 100 then processes (step 606) the left input channel and the right input channel of the selected signal pair according to the cross-channel characteristics to generate a left center channel and a right center channel. In some embodiments, a processor (PROC) as illustrated in FIGS. 3A to 3B and 4A to 4B is used to process the left and right input sounds of a signal pair Road. In some embodiments, PROC includes one or more modules as shown in the fourth diagram A.

Next, the data processing system 100 combines each of the left center channel and the right center channel with a third input channel of the multi-channel audio input signal (step 608) to form a two-channel audio output signal. For example, as shown in Figure 3A, the left center channel and the right center channel (for example: L / R signal pair or Ls / Rs signal pair) processed by the respective PROC are connected to the center channel C and / Or Low Frequency Effect Channel (LFE) combined to produce a two-channel audio output signal L '/ R'. Similarly, as shown in Figure 3B, the left center channel and the right center channel processed by the respective PROC (for example: L / R signal pair, Lss / Rss signal pair or Lrs / Rrs signal pair) are related to The center channel C and / or the low frequency effect channel (LFE) are combined to produce a two-channel audio output signal L '/ R'.

In acoustic engineering, the sound field is usually defined as the area between the leftmost perceptible position and the rightmost perceptible position of audio reproduction. In other words, the sound field is the farthest range that a sound object can be perceived. Therefore, the width of the sound field is defined as the distance between the left and right boundaries. In general, the width of the sound field reproduced in stereo is the distance between the two speakers. In this case, the concept of sound field is applied to each set of symmetrical channels separately. For example, in some embodiments, the data processing system 100 uses a width controller (eg, width controller 434) to further adjust (step 610) the width of a sound field associated with the left center channel and the right center channel , And then combine the left center channel, the right center channel and the third input channel. In some embodiments, the data processing system 100 receives (step 612) a user input that clearly determines the width of the sound field of the two-channel audio output signal. User input can be received on the user interface 500 as shown in the fifth figure.

In some embodiments, the step 606 of processing the left input channel and the right input channel further includes extracting (step 614) a central signal component, a left signal component, and a right side from the left input channel and the right input channel Signal component. For example, as illustrated in the fourth diagram A, the input signal pair 410 is processed by the M / S mixer 422 to generate a center component 426 and a left component and a right component S 424. In some embodiments, the data processing system 100 processes (step 616) the left component and the right component, and then combines the two with the center component to generate a left center channel and a right center channel.

In some embodiments, the step 606 of processing the left input channel and the right input channel further includes performing (step 618) equalization processing on the left component and the right component using a bandpass filter (e.g. The equalizer module 428 executes) to obtain a left band-pass filtered component and a right band-pass filtered component (such as the output signal 450 of the equalizer). In some embodiments, the equalization processing step further generates (step 620) a left residual component based on the difference between the left component and the left bandpass filtered component, and generates a right residual based on the difference between the right component and the right bandpass filtered component Components, such as the left residual component and the right residual component 452 illustrated in the fourth A diagram.

In some embodiments, after the equalization process is performed on the left component and the right component, the data processing system 100 performs the left band pass filter component and the right band pass filter component (for example, generated by the equalizer 428 in the fourth A diagram) Performing (step 622) a first dynamic range compression (for example, by the dynamic range compressor 430 of the fourth image A) separately to obtain a corresponding left compression component and a right compression component.

In some embodiments, after performing the first dynamic range compression, the data processing system 100 executes the left compression component and the right compression component (such as generated by the dynamic range compressor 430 of the fourth A image) (step 624) Crosstalk cancellation (for example, performed by the crosstalk cancellation module 432 in the fourth diagram A) to obtain a crosstalk cancellation left component and a crosstalk cancellation right component.

In some embodiments, the data processing system 100 combines (step 626) crosstalk cancellation left component and crosstalk cancellation right component, left residual component and right residual component, and center component to generate a left center channel and a right center channel. In some embodiments, the combining step further includes: after adjusting (step 628) the width of a sound field associated with the left center channel and the right center channel (for example, adjusted by the width controller 434 of the fourth image A), Then combine the two with the third input channel. For example, as shown in the fourth B diagram, the left and right middle channels generated by the respective PROC are transmitted to the respective width controllers to adjust the width of the sound field. The adjusted signal is combined with a third input channel (eg, C or LFE channel) to generate a stereo output signal 488.

In some embodiments, after adjusting the width of the sound field, the data processing system 100 performs (step 630) a second dynamic range compression (for example, the dynamic range compressor 436 of the fourth image A) to generate the left center channel And the right middle channel.

Finally, it should be noted that the present invention can be implemented in a complete hardware configuration, a complete software configuration, or a configuration including hardware and software components. In a preferred embodiment, the present invention is implemented on software, which includes but is not limited to firmware, resident software, microcode, and so on.

In addition, the present invention can be a computer program product, which is accessible through a computer or computer readable medium, and provides program code that can be used by or related to a computer or any command execution system. For the purposes of this specification, a computer-usable or computer-readable medium can be any tangible device that can include, store, communicate, propagate, or transmit for use by or in connection with an instruction execution system, device, or component Program.

This medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or device or component) or a propagation medium. Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer hard drives, random access memory (RAM), read-only memory (ROM), hard drives, and optical drives. Current examples of optical discs include read-only CD-ROM, readable and writable CD-R / W and DVD.

A data processing system suitable for storing and / or executing program code includes at least one processor, which is directly or indirectly coupled to the memory element through a system bus. The memory component may include local memory, mass storage, and cache memory used during the actual execution of the code. The cache memory may provide temporary storage of at least part of the code to reduce The number of times the volume memory has read the code. In some embodiments, the data processing system is implemented in the form of a semiconductor chip (for example, a system single chip), which integrates all components of a computer or other electronic systems into a single chip substrate.

Input / output (I / O) devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intermediary I / O controllers.

The network adapter can also be connected to the system so that the data processing system can be connected to other data processing systems, remote printers or storage devices through intermediary private or public networks. Modems, cable modems, and Ethernet cards are some common types of network adapters.

The description of the present invention is presented for illustrative and explanatory purposes, but it is not intended to be exhaustive or to limit the disclosed forms of the present invention. Many modifications and changes will be apparent to those of ordinary skill in the art. The purpose of selecting and describing the embodiments is to properly explain the principles and practical applications of the invention, and to enable those with ordinary knowledge in other technical fields to understand the present invention through many embodiments and understand that the present invention can be carried out in accordance with the specific purpose desired. Grooming.

The terms used in the description of the embodiments herein are only used to describe specific embodiments, rather than to limit the scope of patent applications. The singular terms "a", "an", and "the" used in the description of the embodiments and the appended patent applications also cover the plural, unless the context clearly indicates other meanings. The term "and / or" herein refers to and includes all possible combinations of one or more of the listed items. It should be further understood that the terms "comprising" and / or "including" in this specification refer to the existence of the described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more Other features, wholes, steps, operations, elements, components and / or assemblies.

It should also be understood that although words such as first, second, etc. may be used herein to describe different elements, these elements should not be limited by these words. These words are intended to distinguish one element from other elements. For example, a first port may be called a second port. Similarly, a second port may be called a first port without departing from the scope of the embodiment. Both the first port and the second port are ports, but not the same port.

After reading the above detailed description and the teachings presented in the related drawings, those skilled in the art will be able to think of many modifications and alternative implementations of the embodiments disclosed herein. Therefore, it should be understood that the scope of patent application in this case is not limited to the specific embodiments disclosed, and many modifications and other embodiments should also be included in the scope of the attached patent application. Although some specific words are used in this article, they are only general and narrative and are not intended to be limiting.

The purpose of the selection and description of the embodiments is to properly explain the principles of the invention and its practical application, so that other people familiar with the technology can properly use the principles of the present invention and many implementations, and make many modifications according to the specific uses desired.

Claims (42)

    A computer-implemented method for processing a multi-channel input audio signal includes: executing on a computing device, the computing device having one or more processors, a memory, and stored in the memory by the one or Multiple program modules executed by multiple processors: selecting a left input channel and a right input channel from the multi-channel input audio signal, wherein the left input channel and the right input channel correspond to a pair of spaces An upper symmetrical signal source; generating one or more cross-channel features from the left input channel and the right input channel; processing the left input channel and the right input channel according to the cross-channel features to generate A left center channel and a right center channel; and combining the left center channel and the right center channel with a third input channel of the multi-channel input audio signal to form a two-channel output audio signal.         The computer-implemented method described in item 1 of the patent application scope further includes: before combining with the third input channel, first adjusting the width of a sound field associated with the left center channel and the right center channel.         The computer-implemented method as described in item 2 of the patent application scope further includes: receiving a user input indicating the width of the sound field of the two-channel output audio signal.         The computer-implemented method as described in item 1 of the patent application scope, wherein the step of processing the left input channel and the right input channel further includes: extracting a central component from the left input channel and the right input channel, a A left component and a right component; and processing the left component and the right component, and then combining the two with the center component to generate the left center channel and the right center channel.         The computer-implemented method as described in item 4 of the patent application scope, wherein the step of processing the left component and the right component further includes: performing an equalization process on the left component and the right component using a band-pass filter to obtain a left component A bandpass filtered component and a right bandpass filtered component; and based on the difference between the left component and the left bandpass filter, a left residual component is generated, and based on the difference between the right component and the right bandpass filtered component, a right side is generated Residual component.         The computer-implemented method as described in item 5 of the patent application scope further includes: after performing equalization processing on the left component and the right component, performing a first on the left bandpass filter component and the right bandpass filter component, respectively A dynamic range compression to obtain a corresponding left compressed component and a right compressed component.         The computer-implemented method as described in item 6 of the patent application scope further includes: after performing the first dynamic range compression, performing crosstalk cancellation on the left compression component and the right compression component, respectively, to obtain a crosstalk cancellation left The component and a crosstalk eliminate the right component.         The computer-implemented method as described in item 7 of the patent application scope further comprises: combining the left cross-talk cancellation component with the right cross-talk cancellation component, the left residual component and the right residual component, and the central component to generate the left The center channel and the right center channel, wherein the combining step further includes: adjusting the width of a sound field associated with the left center channel and the right center channel before combining with the third input channel.         The computer implementation method as described in item 8 of the patent application scope further includes: after adjusting the width of the sound field, performing a second dynamic range compression to generate the left center channel and the right center channel.         The computer implementation method as described in item 1 of the patent application scope, wherein the left input channel is a left front channel and the right input channel is a right front channel.         The computer implementation method as described in item 1 of the patent application scope, wherein the left input channel is a left surround channel and the right input channel is a right surround channel.         The computer implementation method as described in item 1 of the patent application scope, wherein the left input channel is a left rear surround channel and the right input channel is a right rear surround channel.         The computer implementation method as described in item 1 of the patent application scope, wherein the third input channel is a center channel.         The computer-implemented method as described in item 1 of the patent application scope, wherein the third input channel is a low-frequency effect channel.         An arithmetic device for processing a multi-channel input audio signal, the arithmetic device includes: one or more processors; a memory; and a plurality of program modules, which are stored in the memory and are stored by the one or more Processor execution, wherein when the plurality of program modules is executed by the one or more processors, the computing device is caused to perform a plurality of steps, including: selecting a left input channel from the multi-channel input audio signal and A right input channel, wherein the left input channel and the right input channel correspond to a pair of spatially symmetrical signal sources; one or more cross-channel features are generated from the left input channel and the right input channel ; Processing the left input channel and the right input channel according to the cross-channel features to produce a left center channel and a right center channel; and the left center channel and the right center channel respectively with A third input channel of the multi-channel input audio signal is combined to form a two-channel output audio signal.         The computing device as described in item 15 of the patent application scope, wherein the computing device further executes: before combining with the third input channel, first adjusts a sound field associated with the left center channel and the right center channel width.         The computing device as described in item 16 of the patent application scope, wherein the computing device further executes: receiving a user input indicating the width of the sound field of the two-channel output audio signal.         The computing device according to item 15 of the patent application scope, wherein the step of processing the left input channel and the right input channel further includes: extracting a central component and a left side from the left input channel and the right input channel Component and a right component; and process the left component and the right component, and then combine the two with the center component to generate the left center channel and the right center channel.         The arithmetic device as described in item 18 of the patent application range, wherein the step of processing the left component and the right component further comprises: performing an equalization process on the left component and the right component using a band-pass filter to obtain a left band A pass filter component and a right band pass filter component; and based on the difference between the left component and the left band pass filtered component, a left residual component is generated, and based on the difference between the right component and the right band pass filtered component, a right side is generated Residual component.         The arithmetic device as described in item 19 of the patent application scope, wherein the arithmetic device further executes: after performing equalization processing on the left component and the right component, the left bandpass filter component and the right bandpass filter component are respectively Perform a first dynamic range compression to obtain a corresponding left compressed component and a right compressed component.         The computing device according to item 20 of the patent application scope, wherein the computing device further executes: after performing the first dynamic range compression, performing crosstalk cancellation on the left compressed component and the right compressed component, respectively, to obtain a string The sound eliminates the left component and a string of sounds eliminates the right component.         The computing device as described in item 21 of the patent application scope, wherein the computing device further executes: combining the left component of the crosstalk cancellation with the right component of the crosstalk cancellation, the left residual component and the right residual component, the central component, to Generating the left center channel and the right center channel, wherein the combining step further includes: adjusting the width of a sound field associated with the left center channel and the right center channel before combining with the third input channel .         The arithmetic device as described in Item 22 of the patent application scope, wherein the arithmetic device further executes: after adjusting the width of the sound field, performs a second dynamic range compression to generate the left center channel and the right center channel.         The computing device according to item 15 of the patent application scope, wherein the left input channel is a left front channel and the right input channel is a right front channel.         The computing device as described in item 15 of the patent application scope, wherein the left input channel is a left surround channel and the right input channel is a right surround channel.         The computing device as described in item 15 of the patent application scope, wherein the left input channel is a left rear surround channel, and the right input channel is a right rear surround channel.         The arithmetic device as described in item 15 of the patent application scope, wherein the third input channel is a center channel.         The arithmetic device as described in item 15 of the patent application scope, wherein the third input channel is a low-frequency effect channel.         A computer program product stored in a non-transitory computer readable storage medium, the non-transitory computer readable storage medium is connected to a computing device having one or more processors to process an audio signal, The computer program product includes a plurality of program modules. When the plurality of program modules are executed by the one or more processors, the computing device is caused to perform a plurality of steps, including: selecting one from multi-channel input audio signals Left input channel and a right input channel, wherein the left input channel and the right input channel correspond to a pair of spatially symmetrical signal sources; one or more are generated from the left input channel and the right input channel Cross-channel features; processing the left input channel and the right input channel according to the cross-channel features to produce a left center channel and a right center channel; and the left center channel and the right channel The middle channel is combined with a third input channel of the multi-channel input audio signal to form a two-channel output audio signal.         The computer program product as described in item 29 of the patent application scope, in which the computing device further executes: before combining with the third input channel, first adjust a sound field associated with the left center channel and the right center channel width.         The computer program product as described in item 30 of the patent application scope, in which the computing device further executes: receiving a user input, the user input indicating the width of the sound field of the two-channel output audio signal.         The computer program product as described in item 29 of the patent application scope, wherein the step of processing the left input channel and the right input channel further includes: extracting a central component from the left input channel and the right input channel, a A left component and a right component; and processing the left component and the right component, and then combining the two with the center component to generate the left center channel and the right center channel.         The computer program product as described in item 32 of the patent application range, wherein the step of processing the left component and the right component further comprises: performing an equalization process on the left component and the right component using a band-pass filter to obtain a left A bandpass filtered component and a right bandpass filtered component; and based on the difference between the left component and the left bandpass filtered component, a left residual component is generated, and based on the difference between the right component and the right bandpass filtered component, a The residual component on the right.         The computer program product as described in item 33 of the patent application scope, wherein the arithmetic device further executes: after performing equalization processing on the left component and the right component, the left band-pass filtered component and the right band-pass filtered component A first dynamic range compression is performed respectively to obtain a corresponding left compression component and a right compression component.         The computer program product as described in item 34 of the patent application scope, wherein the computing device further executes: after performing the first dynamic range compression, performing crosstalk cancellation on the left compressed component and the right compressed component, respectively, to obtain a Crosstalk eliminates the left component and a crosstalk eliminates the right component.         The computer program product as described in item 35 of the patent application scope, wherein the computing device further performs: combining the left component of the crosstalk cancellation with the right component of the crosstalk cancellation, the left residual component and the right residual component, and the central component, to Generating the left center channel and the right center channel, wherein the combining step further includes: before combining with the third input channel, first adjusting a sound field associated with the left center channel and the right center channel width.         The computer program product as described in item 36 of the patent application scope, wherein the computing device further executes: after adjusting the width of the sound field, performs a second dynamic range compression to generate the left center channel and the right center channel .         The computer program product as described in item 29 of the patent application scope, wherein the left input channel is a left front channel and the right input channel is a right front channel.         The computer program product as described in item 29 of the patent application scope, wherein the left input channel is a left surround channel and the right input channel is a right surround channel.         The computer program product as described in item 29 of the patent application scope, wherein the left input channel is a left rear surround channel and the right input channel is a right rear surround channel.         The computer program product as described in item 29 of the patent application scope, wherein the third input channel is a center channel.         The computer program product as described in item 29 of the patent application scope, wherein the third input channel is a low-frequency effect channel.