CN115206332A

CN115206332A - Sound effect processing method and device, electronic equipment and storage medium

Info

Publication number: CN115206332A
Application number: CN202110390292.4A
Authority: CN
Inventors: 廖健平
Original assignee: Actions Technology Co Ltd
Current assignee: Actions Technology Co Ltd
Priority date: 2021-04-12
Filing date: 2021-04-12
Publication date: 2022-10-18

Abstract

The embodiment of the application discloses a sound effect processing method, a sound effect processing device, electronic equipment and a storage medium, wherein the method comprises the following steps: determining left sound phase information of a left channel signal and right sound phase information of a right channel signal in the input sound signals; processing the left panning information and the right panning information to obtain a center component and two side components of the input sound signal, wherein the center component represents a similar panning of the left panning information and the right panning information, and the two side components represent a difference panning of the left panning information and the right panning information; performing center filtering processing on the center component, and performing two-side filtering processing on the two-side components; and synthesizing the filtered central component and the filtered two-side component by using the surrounding intensity of the input sound signal to obtain a synthesized sound signal. When using the earphone broadcast synthesis sound signal, sound more tends to the direction placed in the middle, and the both sides characteristic of both sides sound is more obvious, and is more clear during the broadcast, more like the broadcast effect of a pair of audio amplifier of setting in the user dead ahead.

Description

Sound effect processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of signal processing technologies, and in particular, to a method and an apparatus for processing sound effect, an electronic device, and a storage medium.

Background

There are many playing modes of sound, for example, music is played through a sound box, and the sound is emitted just in front of the user; the same music is played through the earphone, and because the music occurs beside the two ears, the sound is confined to a narrow area inside the head and is greatly different from the real sense of hearing.

In general, even if the user wants to play with headphones, the user will feel the same sound as a pair of speakers placed right in front of the user. In the Related art, in order to achieve the above effect, usually, HRTF (Head Related Transfer Function) conversion is performed on the sound of each input channel, so as to simulate the sound production effect of two speakers, and finally, the sound is played through headphones, so that the sound widening and zooming effect can be achieved.

However, the HRTF transformation process only has the widening and zooming effect on sound, and the difference from the real auditory sense still exists.

Disclosure of Invention

The embodiment of the application provides a sound effect processing method and device, electronic equipment and a storage medium, and is used for realizing that when sound is played through an earphone, the center sound is more inclined to the center direction, the characteristics of the two sides of the sound on the two sides are more obvious, and the sound effect is more approximate to the real sense of hearing.

In a first aspect, an embodiment of the present application provides a method for processing sound effects, including:

determining left sound phase information of a left channel signal and right sound phase information of a right channel signal in the input sound signals;

processing the left panning information and the right panning information to obtain a center component and two side components of the input sound signal, wherein the center component represents a similar panning of the left panning information and the right panning information, and the two side components represent a difference panning of the left panning information and the right panning information;

performing centering filtering processing on the centering component, and performing two-side filtering processing on the two-side components;

and synthesizing the filtered central component and the filtered two side components by using the surrounding intensity of the input sound signal to obtain a synthesized sound signal.

According to the embodiment of the application, an input sound signal is processed into a central component and two side components, and the central component and the two side components are respectively subjected to filtering processing, so that the space effect during sound playing is enhanced; and in the component synthesis process, the surrounding intensity is considered, the relation between the component in the middle after filtering and the component on the two sides after filtering in the synthesis process can be flexibly adjusted, so that the sound in the middle is more prone to the direction in the middle when the obtained synthesized sound signal is played, and the characteristics of the two sides of the sound on the two sides are more obvious, so that the playing effect is clearer and is more like the playing effect of a pair of sound boxes arranged in the front of a user.

In some exemplary embodiments, the synthesizing, by applying the surround intensity of the input sound signal and synthesizing the filtered center component and the filtered side components, a synthesized sound signal includes:

determining a first synthesis ratio according to the surrounding intensity of the input sound signal;

and obtaining the synthesized sound signal based on the first synthesis proportion pair of the filtered central component and the filtered two-side component.

According to the embodiment, the surrounding intensity is different, the determined first synthesis proportion is different, the first synthesis proportion directly influences the playing effect, and therefore the synthesis proportion of the middle component after filtering and the two side components after filtering in the synthesis process is flexibly adjusted according to the surrounding intensity, so that the middle sound of the obtained synthesized sound signal is more inclined to the middle direction, the two side characteristics of the two side sounds are more obvious, and the playing effect is clearer.

In some exemplary embodiments, obtaining the synthesized sound signal based on the first synthesis ratio pair of the filtered middle component and the filtered two-side component includes:

synthesizing the filtered central component and the filtered two-side component according to the first synthesis proportion to obtain a first synthesized sound signal;

performing a second synthesis operation by applying the first synthesized sound signal, the center component after reverberation and the two side components after reverberation to obtain a synthesized sound signal;

the center component after reverberation is obtained by performing center reverberation processing on the center component; and the two side components after reverberation are obtained by performing two-side reverberation processing on the two side components.

In the above embodiment, after the first synthesized sound signal is obtained, the second synthesis operation is performed on the center component after the reverberation processing and the two side components after the reverberation processing, so that the addition of the reverberation processing enhances the surround feeling of the virtual surround.

In some exemplary embodiments, the reverberated two-sided components include a left reverberated reference left channel component and a right reverberated reference right channel component;

the second synthesizing operation is performed by applying the first synthesized sound signal, the center component after the reverberation and the two side components after the reverberation to obtain a synthesized sound signal, and the second synthesizing operation includes:

determining a left channel signal and a right channel signal in the first synthesized sound signal;

and synthesizing the center component after the reverberation, the reference left channel component after the left reverberation and the reference right channel component after the right reverberation with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal.

In the above embodiment, in the second synthesis operation, the left channel signal and the right channel signal in the first synthesized sound signal are synthesized with the respective reverberated components, so that the proportion of the respective reverberated components in the synthesis process can be adjusted to enhance the surround feeling of the virtual surround.

In some exemplary embodiments, the two-side reverberation processing is performed on the two-side components to obtain the reverberated two-side components as follows:

performing decorrelation processing on the two side components to obtain a reference left channel component and a reference right channel component;

performing left reverberation processing on the reference left channel component and performing right reverberation processing on the reference right channel component;

and determining the reference left channel component after the left reverberation and the reference left channel component after the right reverberation as two side components after the reverberation.

In the above embodiment, in the reverberation process, the reference left channel component and the reference right channel component obtained after decorrelation of the two side components are respectively subjected to reverberation processing, so that the reference left channel component after left reverberation and the reference right channel component after right reverberation are respectively applied in the subsequent synthesis, so as to make the two side characteristics of the synthesized sound signal more obvious.

In some exemplary embodiments, synthesizing the center component after reverberation, the reference left channel component after left reverberation, and the reference right channel component after right reverberation with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal includes:

processing the center component after reverberation according to a second synthesis proportion, processing a reference left channel component after left reverberation according to a third synthesis proportion, processing a reference right channel component after right reverberation according to a fourth proportion and a first time delay, and synthesizing the reference right channel component after right reverberation with a left channel signal in the first synthesized sound signal;

processing the center component after reverberation according to the second synthesis proportion, processing the reference right channel component after right reverberation according to a fourth synthesis proportion, and synthesizing the reference left channel component after left reverberation with a right channel signal in the first synthesized sound signal according to a third synthesis proportion and a second time delay;

determining the synthesized left channel sound signal and the synthesized right channel sound signal as synthesized sound signals.

In some exemplary embodiments, the performing the centering filtering process on the centering component includes:

acquiring a set centering configuration parameter of the centering component under a first set acoustic image;

determining a target centering filter parameter matched with the set centering configuration parameter based on the set centering configuration parameter and the corresponding relation between the centering configuration parameter and the centering filter parameter;

applying the target centering filter parameter to filter the centering filter component;

the performing of the two-side filtering process on the two-side components includes:

acquiring set two-side configuration parameters of the two-side components under a second set sound image;

and determining the target two-side filter parameters matched with the set two-side configuration parameters based on the set two-side configuration parameters and the corresponding relation between the two-side configuration parameters and the two-side filter parameters.

In a second aspect, an embodiment of the present application provides an audio processing apparatus, including:

the information determining module is used for determining left sound phase information of a left channel signal and right sound phase information of a right channel signal in the input sound signals;

a first processing module, configured to process the left panning information and the right panning information to obtain a center component and two side components of the input sound signal, where the center component represents a similar panning of the left panning information and the right panning information, and the two side components represent a difference panning of the left panning information and the right panning information;

the second processing module is used for performing center filtering processing on the center component and performing two-side filtering processing on the two-side components;

and the synthesis module is used for synthesizing the filtered center component and the filtered two side components by applying the surrounding intensity of the input sound signal to obtain a synthesized sound signal.

In some exemplary embodiments, the synthesis module is specifically configured to:

performing a second synthesis operation by using the first synthesized sound signal, the center component after reverberation and the two side components after reverberation to obtain a synthesized sound signal;

the synthesis module is specifically configured to:

and synthesizing the center component after reverberation, the reference left channel component after left reverberation and the reference right channel component after right reverberation with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal.

In some exemplary embodiments, the apparatus further includes a reverberation module, configured to perform two-side reverberation processing on the two-side components to obtain two-side components after reverberation as follows:

processing the center component after reverberation according to a second synthesis proportion, processing the reference left channel component after left reverberation according to a third synthesis proportion, processing the reference right channel component after right reverberation according to a fourth proportion and a first time delay, and synthesizing the reference right channel component after right reverberation with a left channel signal in the first synthesized sound signal;

In some exemplary embodiments, the second processing module is configured to:

applying the target centered filter parameter to filter the centered filter component;

and determining the target two-side filtering parameters matched with the set two-side configuration parameters based on the set two-side configuration parameters and the corresponding relation between the two-side configuration parameters and the two-side filtering parameters.

In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of any one of the methods when executing the computer program.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of any of the methods described above.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.

Fig. 1 is a schematic diagram of a sound box playing sound effect in the prior art;

FIG. 2 is a diagram illustrating a prior art earphone playing sound effect;

fig. 3 is a schematic diagram illustrating a sound effect played by an earphone according to an embodiment of the present application;

fig. 4 is a schematic diagram illustrating another sound effect played by a headset according to an embodiment of the present application;

fig. 5 is a schematic flowchart illustrating a sound effect processing method according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a sound signal processing according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a sound effect processing apparatus according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

For convenience of understanding, terms referred to in the embodiments of the present application are explained below:

(1) Binaural principle: when stereo sound is implemented, two loudspeakers at an angle to each other are placed in space, each loudspeaker being supplied with a signal from a separate channel. And the signal of each channel is processed at the time of recording: the processing principle is to simulate the biological principle of human ears when the human ears hear sound in nature (human ears can judge the specific position of a sound source according to the phase difference of left ears and right ears when the human ears hear sound), and shows that basically, two sound channel signals are different in phase on a circuit, so that when the human ears hear sound at the intersection of the axes of two loudspeakers, the human ears can feel the effect of stereo.

(2) Acoustic image: the spatial positions of the sound parts, which are shown in the listening sensation of the listener, form an acoustic picture, i.e. the area involved in the sound reproduction, which is usually referred to as the sound image.

(3) Acoustic phase: sound is generated by the vibration of an object, sound waves are a periodic motion, and the precise position that sound waves achieve in a periodic motion is called the phase. Usually in degrees of a circle. The starting point of any wave is 360 degrees away from the starting point of its adjacent wave, that is, all the peaks or troughs are in phase, the peaks and troughs are opposite to each other, and the phase difference is exactly 180 °.

(4) Sound intensity: the method is characterized in that the method refers to the magnitude of the average energy flow density of sound waves, the sound intensity is determined by the magnitude of vibration amplitude, and decibels are measuring units for measuring the quantity ratio of two same units and are mainly used for measuring the sound intensity.

(5) Reverberation: when the sound wave propagates indoors, the sound wave is reflected by obstacles such as walls, ceilings, floors and the like, and the sound wave is absorbed by the obstacles once. Therefore, when the sound source stops sounding, the sound waves are reflected and absorbed for many times in the room and disappear at last, and a plurality of sound waves are mixed for a period of time after the sound source stops sounding, namely, the sound continuation phenomenon still exists after the sound source stops sounding, and the phenomenon is called reverberation.

Any number of elements in the drawings are by way of example and not by way of limitation, and any nomenclature is used for distinction only and not by way of limitation.

There are many playing modes of sound, for example, music is played through a sound box, and the sound is emitted just in front of the user; while the same music is played through the headphones, it sounds like a narrow area confined inside the head, as it occurs near both ears.

The effect of the above-described earphones, which are played out as if they were confined to a narrow area inside the head, is greatly different from the real sense of hearing. In general, even if the user wants to play the sound with the headphones, the sound will sound like a pair of speakers installed right in front of the user. In the related art, in order to achieve the above effect, HRTF conversion is usually performed on the input sound of each channel, so as to simulate the sounding effect of two speakers, and finally, the sound is played through headphones, so that the widening and zooming effect of the sound can be achieved.

However, in the HRTF conversion process, on one hand, only the sound is widened and zoomed out, and the difference between the sound widening effect and the sounding effect of the sound box is still large, and on the other hand, the conversion orders are very high, generally speaking, hundreds of orders are needed, and the highest order can reach thousands of orders, so that the computation amount is very large, and the power consumption is very high. In addition, the order of the conversion is very large, which causes a problem of very large delay, and the large delay also affects the playing effect of the earphone.

Taking the sound recorded by the two-channel as an example, if the sound is played through a pair of speakers, the playing effect is generally set right in front of the user, as shown in fig. 1; 11 is the first sound image, i.e. the region in which the sound plays in this example. The same sound, played through the headphones, will play as if it were confined to a narrow area inside the head due to the sound location near the ears, as shown in fig. 2, with 21 being a second sound image, i.e., the area involved in the sound playing in this example. In practical applications, even if the user wants to play the sound with headphones, the sound will sound like a pair of speakers arranged right in front of the user, as shown in fig. 3, 31 is a third sound image, that is, the area involved in the sound playing in this example. In addition, if the sound is clearer, for example, the playing area of the central sound is more inclined to the central direction, and the playing areas of the two side sound volumes are more inclined to the two sides, referring to fig. 4, where 41 is the third sound image, i.e., the playing area of the central sound, 42 is the fourth sound image, and 43 is the fifth sound image, i.e., the playing areas of the two side sounds, such sound playing effect is more clear as if a pair of speakers is provided right in front of the user.

Therefore, the sound effect processing method is provided, the filtering and artificial reverberation technology is combined, the center sound and the two-side sound are flexibly prepared, the sound recorded by the two channels is played by the earphone, the effect of the sound recorded by the two channels is similar to the sound produced by a pair of sound boxes arranged right in front of a user, meanwhile, the power consumption is saved, and the time delay is reduced.

After introducing the design concept of the embodiment of the present application, some simple descriptions are provided below for application scenarios to which the technical solution of the embodiment of the present application can be applied, and it should be noted that the application scenarios described below are only used for describing the embodiment of the present application and are not limited. In specific implementation, the technical scheme provided by the embodiment of the application can be flexibly applied according to actual needs.

The method can be applied to electronic equipment, if the electronic equipment has an audio playing function, such as an earphone, the sound output after being processed by the method can realize the effect of sounding a pair of sound boxes arranged right in front of a user; if the electronic equipment does not have the audio playing function, the sound processed by the method can be played through a common earphone, and the effect of sounding a pair of sound boxes arranged right in front of the user can also be realized. Therefore, in the embodiment of the present application, the electronic device is described by taking an earphone as an example.

Of course, the method provided in the embodiment of the present application is not limited to be used in the application scenario shown in fig. 1, and may also be used in other possible application scenarios, and the embodiment of the present application is not limited. The functions that can be implemented by each device in the application scenario shown in fig. 1 will be described in the following method embodiments, and will not be described in detail herein.

To further illustrate the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the accompanying drawings and the detailed description. Although the embodiments of the present application provide method steps as shown in the following embodiments or figures, more or fewer steps may be included in the method based on conventional or non-inventive efforts. In steps where no necessary causal relationship exists logically, the order of execution of the steps is not limited to that provided by the embodiments of the present application.

Referring to fig. 5, an embodiment of the present application provides a method for processing sound effects, including the following steps:

s501, determining left sound phase information of a left channel signal and right sound phase information of a right channel signal in the input sound signals.

S502, processing the left vocal phase information and the right vocal phase information to obtain a center component and two side components of the input sound signal, wherein the center component represents a similar vocal phase of the left vocal phase information and the right vocal phase information, and the two side components represent a difference vocal phase of the left vocal phase information and the right vocal phase information.

And S503, performing center filtering processing on the center component, and performing two-side filtering processing on the two-side components.

S504, synthesizing the filtered center component and the filtered two side components by using the surrounding intensity of the input sound signal to obtain a synthesized sound signal.

According to the embodiment of the application, an input sound signal is processed into a central component and two side components, and the central component and the two side components are respectively subjected to filtering processing, so that the space effect during sound playing is enhanced; and in the component synthesis process, the surrounding strength is considered, the relation between the central component after filtering and the two side components after filtering in the synthesis process can be flexibly adjusted, so that the obtained synthesized sound signal is in playing, the central sound is more inclined to the central direction, and the characteristics of the two sides of the two side sounds are more obvious, therefore, the playing effect is clearer, and the playing effect is more like that of a pair of sound boxes arranged in the front of a user.

In order to flexibly adjust the ratio of the center sound and the side sound, the input sound signal is processed first, and the center sound and the side sound are extracted, which can be implemented through S501 and S502.

Referring to S501, since the input sound signal is recorded in two channels, after receiving the input sound signal, the left channel information and the right channel information are extracted, and illustratively, the left channel phase in the left channel information is represented by L, and the right channel phase in the right channel information is represented by R.

Referring to S502, in order to represent a center sound and two side sounds in a sound signal, left channel information and right channel information are processed, a similar vocal facies of the left vocal facies information and the right vocal facies information is called a center component, and the center component represents a center sound in the input sound signal; the difference panning of the left panning information and the right panning information is referred to as a two-sided component, which represents two-sided sound in the input sound signal.

Illustratively, the central component may be referred to as Mid signal, and the two Side components as Side signal, that is, mid signal represents the signal with the acoustic phase centered, and Side signal represents the signal with the acoustic phase off center. In a specific example, the Mid signal may be obtained by adding a left channel acoustic phase and a right channel acoustic phase, and the Side signal includes two parts, and may be obtained by subtracting the left channel acoustic phase and the right channel acoustic phase, for example, mid = L + R, side = (L-R) U (R-L), where U is a union symbol. In this way, the center component and both-side components of the input sound signal are obtained, and then the center component and both-side components are processed.

In S503, the center component and the both-side components are respectively subjected to filtering processing, and for distinction, the filtering processing for the center component is referred to as center filtering processing, and the filtering processing for the both-side components is referred to as both-side filtering processing.

For example, the filter applied in the centering filtering process is called a centering filter, and the frequency and loudness value of the centering filter are configured to achieve the effect of centering filtering, and the effect of the centering filtering process is to make the centering component more in the center direction, which can be achieved by a FIR filter (Finite Impulse Response filter). The FIR filter is also called a non-recursive filter, and can ensure any amplitude-frequency characteristic and simultaneously have strict linear phase-frequency characteristics, and the unit sampling response of the FIR filter is finite, so that the filter is a stable system. In addition, the IIR (Infinite Impulse Response filter) may be used, and in general, the IIR filter has lower power consumption and smaller time delay.

During centering filtering processing, acquiring a centering configuration parameter of a centering component under a first set acoustic image; determining a target centering filter parameter matched with the centering configuration parameter based on the centering configuration parameter and the corresponding relation between the centering configuration parameter and the centering filter parameter; the centered filter component is filtered using the target centered filter parameter. Specifically, the first set acoustic image is a preset area involved in the playing of a signal with a desired centering component, and the width and distance of the acoustic image determine the area and size involved in the playing, so the width and distance are collectively referred to as configuration parameters, that is, the set centering configuration parameters (including the set centering configuration width and the set centering configuration distance) capable of generating the first set acoustic image are obtained according to the user requirements or preset. And determining a target centered filtering parameter matched with the set centered configuration parameter according to the preset corresponding relationship between the centered configuration parameter and the centered filtering parameter and the set centered configuration parameter, wherein the target centered filtering parameter exemplarily comprises a target centered filtering frequency and a target centered filtering loudness. In this way, the centered component is centered filtered using the target centered filtering frequency and the target centered filtering loudness.

Similarly, the filter applied during the two-side filtering process is called a two-side filter, and the frequency and loudness of the two-side filter are configured to achieve the two-side filtering effect, and the two-side filtering process has the effect that the two-side characteristics of the two-side components are more obvious, and an FIR filter or an IIR filter can be used.

During the two-side filtering processing, acquiring set two-side configuration parameters of two-side components under a second set sound image; and determining a target two-side filtering parameter matched with the set two-side configuration parameter based on the set two-side configuration parameter and the corresponding relation between the two-side configuration parameter and the two-side filtering parameter. Specifically, the second set acoustic image is a preset area involved in the playing of a signal of a desired two-side component, and the width and distance of the acoustic image determine the area and size involved in the playing, so the width and distance are collectively referred to as configuration parameters, that is, the set two-side configuration parameters (including the set two-side configuration width and the set two-side configuration distance) capable of generating the second set acoustic image are obtained according to the user requirement or preset. And determining a target two-side filter parameter matched with the preset two-side configuration parameter according to the preset corresponding relation between the two-side configuration parameter and the two-side filter parameter and the preset two-side configuration parameter, wherein the target two-side filter parameter exemplarily comprises a target two-side filter frequency and a target two-side filter loudness. Thus, the two-sided component is filtered on both sides using the target two-sided filtering frequency and the target two-sided filtering loudness.

Referring to S504, after the filtering of the center component and the side components is completed, the surround intensity of the input sound signal is applied to synthesize the filtered center component and the filtered side components, so as to obtain a synthesized sound signal.

The synthesis process can be specifically realized by the following modes: determining a first synthesis ratio according to the surround intensity of the input sound signal; and obtaining a synthesized sound signal based on the first synthesis proportion and the filtered central component and the filtered two-side component.

Specifically, the surround intensity (sound intensity) of the input sound signal is detected, and the relationship between the surround intensity and the first synthesis ratio is preset, so that the first synthesis ratio can be determined based on the surround intensity. Illustratively, the first composition ratio includes a median composition ratio and a two-sided composition ratio. In a specific example, for example, when the surround intensity is a, the center synthesis ratio is determined to be 0.8, and the two-Side synthesis ratio is determined to be 0.2, that is, the value Mid is multiplied by 0.8 and the value Side is multiplied by 0.2 to obtain the synthesized sound signal. The synthesized sound signal obtained in the way has the advantages that the center component is more inclined to the center direction, and the characteristics of the two sides of the components on the two sides are more obvious, namely, the sound playing effect is clearer, and the problem of sound blur is effectively solved; the introduction of the surround intensity can also achieve the effect of stereo widening.

In addition, in order to enhance the surround effect and the spatial sense of surrounding, reverberation processing is also applied in the synthesis process. Specifically, the synthesis process includes two synthesis operations, where an operation of synthesizing the filtered middle component and the filtered two side components according to a first synthesis ratio to obtain a first synthesized sound signal is referred to as a first synthesis operation; and then, performing second synthesis operation on the first synthesized sound signal obtained in the first synthesis operation, the center component after reverberation and the two side components after reverberation to obtain a synthesized sound signal.

Since the center component after reverberation and the both-side components after reverberation are applied in the second synthesis process, the reverberation process will be explained.

Firstly, a centering reverberation process is described, a process of reverberating a centering component is referred to as centering reverberation, specifically, the centering component is artificially reverberated, for example, in the reverberation process, reflected sounds in an early reflection stage (which may be a set time period) in an entire reflection process are introduced into a plurality of delay lines to be realized, the delay lines may be referred to as delay lines, and an actual operation is realized by adding a delay duration represented by the delay lines. Compared with the method of realizing reverberation by applying delay line and convolution, the method has the advantages of simpler operation and flexible configuration.

Secondly, explaining the reverberation process of two sides, comparing the reverberation of the two sides with the reverberation of the center, and performing decorrelation on the components of the two sides through a preset decorrelation filter to obtain a reference left channel component and a reference right channel component; performing left reverberation processing on the reference left channel component, and performing right reverberation processing on the reference right channel component; and determining the reference left channel component after the left reverberation and the reference left channel component after the right reverberation as two side components after the reverberation. Specifically, the correlated parts in the two side components can be canceled, and the uncorrelated parts are the respective difference parts in the two side components, so that the two side components are decorrelated, and then the reference left channel component and the reference right channel component are obtained according to the channel characteristics. The process of performing left reverberation processing on the reference left channel component and performing right reverberation processing on the reference right channel component can also be implemented by introducing a plurality of delay lines, and the specific number of the delay lines and the time length of each delay line can be adjusted according to actual requirements.

After the center component after reverberation and the two side components after reverberation are obtained, the second synthesizing operation is explained.

Since the two side components after reverberation include the reference left channel component after left reverberation and the reference right channel component after right reverberation, in this way, during the second synthesis operation, the left channel signal and the right channel signal in the first synthesized sound signal are determined; and synthesizing the center component after reverberation, the reference left channel component after left reverberation and the reference right channel component after right reverberation with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal. The set synthesis rule includes a proportion rule and a delay rule, the proportion rule is, for example, that different components are multiplied by a certain proportion and then added, and the delay rule is, for example, that different components are subjected to certain delay processing and then added.

Specifically, the center component L after reverberation is set _j According to a second synthesis ratio a ₂ Processing and converting the left reverberated reference left channel component L _f According to a third synthesis proportion a ₃ Processing and converting the right reverberated reference right channel component L _r According to a third proportion a ₃ And a first time delay t ₁ Processing, with the left channel signal L in the first synthesized sound signal ₁ Synthesizing, i.e. the synthesized left channel signal L _hf ＝L _j *a ₂ +L _f *a ₃ +L _r *a ₄ (t ₁ )+L ₁ Wherein (t) ₁ ) Indicating that the first time delay process has elapsed. Mixing the reverberated intermediate component L _j According to the second synthesis ratio a ₂ Processing, processing the reference right channel component Lr after the right reverberation according to a fourth synthesis proportion a4, and processing the reference left channel component L after the left reverberation _f According to a third synthesis proportion a ₃ And a second time delay t ₂ Processing with the right channel signal L in the first synthesized sound signal ₂ Synthesizing, i.e. the synthesized right channel signal L _hr ＝L _j *a ₂ +L _r *a ₄ +L _f *a ₃ (t ₂ )+L ₂ (ii) a Determining the synthesized left channel sound signal and the right channel sound signal as synthesized sound signals, wherein,(t ₂ ) Indicating that the second time delay process has been passed. Determining L _hf And L _hr To synthesize a sound signal.

When the synthesized sound signal is played through the earphone, the effect that the synthesized sound signal is played like a pair of sound boxes arranged right in front of a user can be achieved. In addition, in order to make the technical solution of the present application easier to understand, the following describes the whole sound effect processing method with reference to fig. 6.

Taking two synthesis operations as an example, firstly processing an input sound signal to obtain a centering component and two side components, performing centering filtering processing on the centering component to obtain a filtered centering component, and performing centering reverberation processing to obtain a reverberated centering component; and for the two side components, performing two-side filtering processing to obtain filtered two side components, performing two-side reverberation processing, performing decorrelation processing to obtain a reference left channel component and a reference right channel component in the two-side reverberation processing, performing left reverberation processing on the reference left channel component, and performing right reverberation processing on the reference right channel component. In the first synthesis process, determining the synthesis proportion of the filtered central component and the filtered two-side component according to the detected surrounding intensity to obtain a first synthesis signal; in the second synthesis operation, the center component after the center reverberation processing, the reference left channel component after the left reverberation processing, and the reference right channel component after the right reverberation processing are respectively synthesized with the left channel signal and the right channel signal in the first synthesis component, so that the obtained synthesized sound signal is output. Therefore, when the synthesized sound signal is played, the surround sense is stronger, and the sound is clearer.

As shown in fig. 7, based on the same inventive concept as the above sound effect processing method, an embodiment of the present application further provides a sound effect processing apparatus, including: an information determination module 701, a first processing module 702, a second processing module 703 and a composition module 704.

The information determining module 701 is configured to determine left panning information of a left channel signal and right panning information of a right channel signal in an input sound signal;

a first processing module 702, configured to process the left panning information and the right panning information to obtain a center component and two side components of the input sound signal, where the center component represents a similar panning of the left panning information and the right panning information, and the two side components represent a difference panning of the left panning information and the right panning information;

a second processing module 703, configured to perform center filtering on the center component and perform two-side filtering on the two-side components;

and a synthesizing module 704, configured to synthesize the filtered center component and the filtered two-side components by applying the surround intensity of the input sound signal, so as to obtain a synthesized sound signal.

In some exemplary embodiments, the synthesis module 704 is specifically configured to:

determining a first synthesis ratio according to the surround intensity of the input sound signal;

and obtaining a synthesized sound signal based on the first synthesis proportion and the filtered central component and the filtered two-side component.

synthesizing the filtered central component and the filtered two-side component according to a first synthesis proportion to obtain a first synthesized sound signal;

performing a second synthesis operation by using the first synthesized sound signal, the center component after the reverberation and the two side components after the reverberation to obtain a synthesized sound signal;

the synthesis module 704 is specifically configured to:

and synthesizing the center component after reverberation, the reference left channel component after left reverberation and the right channel component after right reverberation with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal.

In some exemplary embodiments, the apparatus further includes a reverberation module, configured to perform two-side reverberation processing on the two-side components to obtain the reverberated two-side components by:

processing the center component after reverberation according to a second synthesis proportion, processing a reference left channel component after left reverberation according to a third synthesis proportion, processing a reference right channel component after right reverberation according to a fourth proportion and a first time delay, and synthesizing the reference right channel component after right reverberation with a left channel signal in a first synthesized sound signal;

processing the center component after reverberation according to a second synthesis proportion, processing a reference right channel component after right reverberation according to a fourth synthesis proportion, processing a reference left channel component after left reverberation according to a third synthesis proportion and a second time delay, and synthesizing the reference left channel component after left reverberation with a right channel signal in the first synthesized sound signal;

In some exemplary embodiments, the second processing module 703 is configured to:

The sound effect processing device and the sound effect processing method provided by the embodiment of the application adopt the same inventive concept, can obtain the same beneficial effects, and are not repeated herein.

Based on the same inventive concept as the sound effect processing method, the embodiment of the present application further provides an electronic device, where the electronic device may be a desktop computer, a portable computer, a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a server, and the like (which may be a control device or a control system inside the smart device, or an external device communicating with the smart device, such as an earphone). As shown in fig. 8, the electronic device may include a processor 801 and a memory 802.

The Processor 801 may be a general-purpose Processor, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware components, and may implement or execute the methods, steps, and logic blocks disclosed in the embodiments of the present Application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of a method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or may be implemented by a combination of hardware and software modules in a processor.

The memory 802, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory may include at least one type of storage medium, which may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charged Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and the like. The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 802 in the embodiments of the present application may also be circuitry or any other device capable of performing a storage function for storing program instructions and/or data.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; the computer storage media may be any available media or data storage device that can be accessed by a computer, including but not limited to: a mobile storage device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)) and various media that can store program codes.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media that can store program codes include a removable Memory device, a Random Access Memory (RAM), a magnetic Memory (e.g., a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical Memory (e.g., a CD, a DVD, a BD, an HVD, etc.), and a semiconductor Memory (e.g., a ROM, an EPROM, an EEPROM, a nonvolatile Memory (NAND FLASH), a Solid State Disk (SSD)).

The above embodiments are only used to describe the technical solutions of the present application in detail, but the above embodiments are only used to help understanding the method of the embodiments of the present application, and should not be construed as limiting the embodiments of the present application. Modifications and substitutions that may be readily apparent to those skilled in the art are intended to be included within the scope of the embodiments of the present application.

Claims

1. A method for processing sound effect is characterized by comprising the following steps:

processing the left panning information and the right panning information to obtain a center component and two side components of the input sound signal, wherein the center component represents similar panning of the left panning information and the right panning information, and the two side components represent difference panning of the left panning information and the right panning information;

performing center filtering processing on the center component, and performing two-side filtering processing on the two-side components;

and synthesizing the filtered central component and the filtered two-side component by using the surrounding intensity of the input sound signal to obtain a synthesized sound signal.

2. The method of claim 1, wherein the applying the surround intensity of the input sound signal to synthesize the filtered center component and the filtered side components to obtain a synthesized sound signal comprises:

3. The method of claim 2, wherein obtaining the synthesized sound signal based on the first synthesis ratio pair of the filtered center component and the filtered side components comprises:

4. The method of claim 3, wherein the reverberated two-sided components include a left reverberated reference left channel component and a right reverberated reference right channel component;

the second synthesizing operation is performed on the first synthesized sound signal, the center component after reverberation and the two side components after reverberation to obtain a synthesized sound signal, and the second synthesizing operation includes:

5. The method of claim 3, wherein the two-side reverberation processing is performed on the two-side components to obtain two-side reverberated components as follows:

6. The method of claim 4, wherein synthesizing the reverberated center component, the left reverberated reference left channel component, and the right reverberated reference right channel component with the left channel signal and the right channel signal in the first synthesized signal according to a set synthesis rule to obtain a synthesized sound signal comprises:

and determining the synthesized left channel sound signal and the synthesized right channel sound signal as synthesized sound signals.

7. The method according to any one of claims 1 to 6, wherein the centering filtering process on the centering component comprises:

8. An apparatus for processing sound effects, comprising:

the second processing module is used for carrying out center filtering processing on the center component and carrying out two-side filtering processing on the two-side components;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.