CN116320908A - Virtual stereo generation method and electronic equipment - Google Patents
Virtual stereo generation method and electronic equipment Download PDFInfo
- Publication number
- CN116320908A CN116320908A CN202310237775.XA CN202310237775A CN116320908A CN 116320908 A CN116320908 A CN 116320908A CN 202310237775 A CN202310237775 A CN 202310237775A CN 116320908 A CN116320908 A CN 116320908A
- Authority
- CN
- China
- Prior art keywords
- signal
- channel
- gain
- channel signal
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000005236 sound signal Effects 0.000 claims abstract description 69
- 238000001914 filtration Methods 0.000 claims description 9
- 230000002238 attenuated effect Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 12
- 230000009286 beneficial effect Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 5
- 238000005070 sampling Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 3
- 230000002688 persistence Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
Abstract
The invention relates to the technical field of audio signal processing, and discloses a virtual stereo generation method and electronic equipment. The generation method comprises the following steps: the method comprises the steps of obtaining a first channel signal and a second channel signal, calculating a center channel signal and a time domain differential signal according to the first channel signal and the second channel signal, generating a reverberation signal according to the center channel signal, generating a surround sound signal according to the time domain differential signal, and generating a virtual stereo signal according to the first channel signal, the reverberation signal and the surround sound signal. The embodiment can generate the virtual stereo signal of which each channel is mixed with reverberation and surround sound based on the original two channel signals, and restore the lost space information of the original channel signals so as to reconstruct a complex sound field, thereby being beneficial to improving the lost space feeling in the recording/playback process.
Description
Technical Field
The invention relates to the technical field of audio signal processing, in particular to a virtual stereo generation method and electronic equipment.
Background
Stereophonic sound refers to sound having a stereoscopic impression. Because the sound source has a defined spatial position, the sound has a defined directional source, and the human hearing has the ability to discern the location of the sound source. Particularly, when a plurality of sound sources are simultaneously sounding, people can sense the position distribution condition of each sound source in space by hearing. In this sense, all sounds made in nature are stereophonic.
At present, when electronic devices such as headphones and stereophonic sound are used for recording sound, although the recorded sound is stereophonic sound, spatial information of the sound is usually lost in the recording process, so that when the sound is played back later, the spatial sense of the sound is insufficient, and the stereophonic effect is poor.
Disclosure of Invention
An object of an embodiment of the present invention is to provide a virtual stereo generating method and an electronic device, which can solve the drawbacks of the related art.
In a first aspect, an embodiment of the present invention provides a method for generating a virtual stereo, including:
acquiring a first sound channel signal and a second sound channel signal;
calculating a center channel signal and a time domain differential signal according to the first channel signal and the second channel signal;
generating a reverberation signal from the center channel signal;
generating a surround sound signal according to the time domain differential signal;
and generating a virtual stereo signal according to the first channel signal, the reverberation signal and the surround sound signal.
Optionally, the first channel signal, the reverberation signal, and the surround sound signal have corresponding gain coefficients, and generating the virtual stereo signal from the first channel signal, the reverberation signal, and the surround sound signal includes:
calculating gain signals corresponding to the first channel signal, the reverberation signal and the surround sound signal according to the first channel signal, the reverberation signal, the surround sound signal and the corresponding gain coefficients;
and fusing the gain signals to generate a virtual stereo signal.
Optionally, the calculating the gain signals corresponding to the first channel signal, the reverberation signal, and the surround sound signal according to the first channel signal, the reverberation signal, the surround sound signal, and the corresponding gain coefficients includes:
calculating a first channel gain signal according to the first channel signal and the leading gain coefficient;
calculating a reverberation gain signal according to the reverberation signal and the central gain coefficient;
and calculating a surround sound gain signal according to the surround sound signal and the space gain coefficient.
Optionally, the fusing each of the gain signals, generating a virtual stereo signal includes:
and summing the first channel gain signal, the reverberation gain signal and the surround sound gain signal to generate a virtual stereo signal.
Optionally, the calculating a center channel signal according to the first channel signal and the second channel signal includes:
and carrying out average processing on the first channel signal and the second channel signal to obtain a center channel signal.
Optionally, the calculating a time domain differential signal according to the first channel signal and the second channel signal includes:
subtracting the second channel signal from the first channel signal to obtain a signal difference value;
dividing the signal difference by a preset coefficient to obtain a time domain differential signal.
Optionally, the generating a reverberation signal according to the center channel signal includes:
performing attenuation processing on the center channel signal to obtain a target attenuation signal;
performing delay processing on the target attenuation signal to obtain a delay signal;
and filtering the mixed signal of the delay signal and the center channel signal according to a preset filter to obtain a reverberation signal.
Optionally, the generating the surround sound signal according to the time domain differential signal includes:
performing Fourier transform on the time domain differential signals to obtain frequency domain differential signals;
calculating a frequency domain gain signal according to the frequency domain differential signal and a preset gain factor;
and performing inverse Fourier transform processing on the frequency domain gain signal to obtain a surround sound signal.
In a second aspect, an embodiment of the present invention provides an electronic device, including:
at least one processor;
and a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
The method for generating the virtual stereo, provided by the embodiment of the invention, comprises the following steps: the method comprises the steps of obtaining a first channel signal and a second channel signal, calculating a center channel signal and a time domain differential signal according to the first channel signal and the second channel signal, generating a reverberation signal according to the center channel signal, generating a surround sound signal according to the time domain differential signal, and generating a virtual stereo signal according to the first channel signal, the reverberation signal and the surround sound signal. The embodiment can generate the virtual stereo signal of which each channel is mixed with reverberation and surround sound based on the original two channel signals, and restore the lost space information of the original channel signals so as to reconstruct a complex sound field, thereby being beneficial to improving the lost space feeling in the recording/playback process.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
Fig. 1 is a schematic structural diagram of an earphone according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a channel distribution according to an embodiment of the present invention;
fig. 3 is a flow chart of a virtual stereo generation method according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of S33 shown in fig. 3;
fig. 5 is a schematic diagram of a flow of generating a reverberation signal according to an embodiment of the present invention;
fig. 6 is a schematic flow chart of S34 shown in fig. 3;
fig. 7 is a schematic structural diagram of a virtual stereo generating apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of the third generation module shown in FIG. 7;
FIG. 9 is a schematic diagram of the first generation module shown in FIG. 7;
FIG. 10 is a schematic diagram of the second generation module shown in FIG. 7;
fig. 11 is a schematic diagram of a hardware structure of a controller according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, if not in conflict, the features of the embodiments of the present invention may be combined with each other, which is within the protection scope of the present invention. In addition, while functional block division is performed in a device diagram and logical order is shown in a flowchart, in some cases, the steps shown or described may be performed in a different order than the block division in the device, or in the flowchart. Furthermore, the words "first," "second," "third," and the like as used herein do not limit the order of data and execution, but merely distinguish between identical or similar items that have substantially the same function and effect.
The electronic device provided by the embodiment of the invention can be electronic devices such as an earphone, a mobile phone, a sound box, a smart watch, a tablet personal computer, a portable audio product and the like, and when the electronic device is the earphone, the earphone can comprise an in-ear earphone, a headphone or an ear-hanging earphone.
Referring to fig. 1, the earphone 100 includes a first transducer 11, a first ADC converter 12, a first sampling rate converter 13, a second transducer 14, a second ADC converter 15, a first sampling rate converter 16, and a controller 17.
The first transducer 11 is used for collecting sound signals, which may include speech signals and ambient sound signals.
The first ADC converter 12 is for converting the sound signal into a digital signal.
The first sample rate converter 13 samples the digital signal according to the sample rate to obtain a first channel signal, where the first channel signal is a signal corresponding to a first channel, the first channel is a channel set in a first sound source direction, the sound source direction is a position of a sound source relative to a listener, the first sound source direction may be a position corresponding to a left ear of the listener, and accordingly, referring to fig. 2, the first channel is a channel set in a left front sound source direction of the listener, the first channel is a left front channel, the first channel signal is a left front channel signal, the first sound source direction may also be a position corresponding to a right ear of the listener, and accordingly, as shown in fig. 2, the first channel is a channel set in a right front sound source direction of the listener, and the first channel signal is a right front channel signal.
The second transducer 14 is used to collect sound signals, which may include speech signals and ambient sound signals. In some embodiments, the sound signals collected by the second transducer 14 and the first transducer 11 are mutually independent sound signals collected at different spatial locations.
The second ADC converter 15 is for converting the sound signal into a digital signal.
The second sample rate converter 16 samples the digital signal according to the sampling rate to obtain a second channel signal, where the second channel signal is a signal corresponding to the second channel, the second channel is a channel set in a second sound source direction, the second sound source direction may be a position corresponding to a left ear of the listener, and accordingly, as shown in fig. 2, the second channel is a channel set in a left front sound source direction of the listener, the second channel is a left front channel, the second channel signal is a left front channel signal, and the second channel direction may also be a position corresponding to a right ear of the listener, and accordingly, as shown in fig. 2, the second channel is a channel set in a right front sound source direction of the listener, the second channel is a right front channel, and the second channel is a right front channel signal. It is understood that when the second channel signal is a left front channel signal, the first channel signal may be a right front channel signal, and when the second channel signal is a right front channel signal, the first channel signal may be a left front channel signal.
The controller 17 generates a virtual stereo signal corresponding to the first channel or a virtual stereo signal corresponding to the second channel according to the first channel signal and the second channel signal in combination with a virtual stereo generation method described below.
The embodiment of the invention provides a virtual stereo generation method. Referring to fig. 3, the virtual stereo generation method S300 includes:
s31, acquiring a first channel signal and a second channel signal;
in this step, the controller obtains a first channel signal and a second channel signal of a frame-by-frame according to a preset sampling frequency, wherein the lengths of the first channel signals of each frame are equal, and the lengths of the second channel signals of each frame are equal.
In some embodiments, the length of the first channel signal per frame is equal to the length of the first channel signal per frame. Wherein, the first channel signal of the nth frame may be expressed as:
data_l(n)＝[data_l(0),data_l(1),···,data_l(PART_LEN-1)] T
where part_len is the length of the first channel signal of one frame, part_len=256.
The second channel signal at the nth frame may be expressed as:
data_r(n)＝[data_r(0),data_r(1),···,data_r(PART_LEN-1)] T
where part_len is the length of the second channel signal of one frame, part_len=256.
S32, calculating a center channel signal and a time domain differential signal according to the first channel signal and the second channel signal;
in this step, the center channel signal is a signal corresponding to the center channel, and referring to fig. 2, the center channel refers to a channel arranged in the direction of the sound source directly in front of the listener, and the controller can simulate the center channel according to the first channel signal and the second channel signal, and calculate the center channel signal corresponding to the center channel.
In some embodiments, after the controller obtains the first channel signal of any frame and the second channel signal of the corresponding frame, the controller performs an average processing on the first channel signal of the frame and the second channel signal of the corresponding frame according to equation one to obtain the center channel signal:
wherein Center is a frame of Center channel signal, and data_l and data_r respectively represent a first channel signal of any frame and a second channel signal of a corresponding frame.
The direction sensing information is mostly included in the "difference" signal, and the "difference" signal may be obtained by a time domain differential signal, and the direction sensing information refers to the spatial azimuth information of the sound source, and the spatial azimuth of the sound source may be azimuth such as right front, left front, right front, left rear, right rear, and the like.
Although the separate first channel signal or the separate second channel signal tends to lose the directional sense information, the controller may calculate a time domain differential signal according to the first channel signal and the second channel signal to extract the directional sense information, compensating for the directional sense information lost by the separate first channel signal or the separate second channel signal.
In some embodiments, the controller subtracts the second channel signal from the first channel signal to obtain a signal difference, and divides the signal difference by a predetermined coefficient to obtain a time domain differential signal.
In some embodiments, the predetermined coefficient is 2, and the controller calculates the time domain differential signal according to equation two:
wherein td is a time domain differential signal of one frame, and data_l and data_r respectively represent a first channel signal of any frame and a second channel signal of a corresponding frame.
S33, generating a reverberation signal according to the center channel signal;
in this step, the reverberation signal is a signal simulating a reverberation phenomenon, where the reverberation phenomenon refers to a phenomenon that when an acoustic wave propagates in a room, the acoustic wave is reflected by an obstacle such as a wall, a ceiling, or a floor, and each reflection is absorbed by the obstacle, and when the acoustic wave stops, the acoustic wave is reflected and absorbed multiple times in the room, and finally disappears, i.e., the sound still existing after the sound of the indoor acoustic wave stops.
The first channel signal alone or the second channel signal alone often lacks spatial depth perception information, which is spatially distributed information of sound sources, which is spatially distributed as a distance between a listener and the sound sources. The present embodiment can compensate for spatial depth sensation information lacking in the separate first channel signal or the separate second channel signal by simulating a reverberation phenomenon from a reverberation signal generated from the center channel signal.
S34, generating a surround sound signal according to the time domain differential signal;
in this step, the surround sound signal is a signal that simulates the full spatial stereo perception of the spatial sound source position by human hearing. In some embodiments, the controller may generate a surround sound signal corresponding to a designated channel from the time domain differential signal, as shown in fig. 2, the designated channel may be a left rear channel for a left front channel, where the left rear Fang Sheng channel corresponding surround sound signal may be referred to as a left rear surround sound signal, and a right rear channel for a right front channel, where the left rear Fang Sheng channel corresponding surround sound signal may be referred to as a right rear surround sound signal.
Therefore, the present embodiment can generate a surround sound signal that is advantageous for a listener to perceive a sense of direction from a time domain differential signal containing the sense of direction information.
And S35, generating a virtual stereo signal according to the first channel signal, the reverberation signal and the surround sound signal.
In this step, the controller may perform a downmix process (downmix) on the first channel signal, the reverberation signal, and the surround sound signal to obtain the virtual stereo signal. The downmix processing is to mix the input signals of more channels and output the mixed signals of fewer channels, as described above, the first channel signal corresponds to the front left channel, the reverberation signal corresponds to the center channel signal, and the surround signal corresponds to the rear left channel or the rear right channel.
In general, electronic devices such as headphones are mostly equipped with dual speakers to play back audio content, so that, based on the application of adapting to a dual speaker playback scene, the played back audio content completely retains all spatial information contained in the multi-channel signal, in some embodiments, the controller may perform a downmix process according to the first channel signal, the reverberation signal, and the surround sound signal to obtain a virtual stereo signal in a dual-channel or stereo format, for example, the controller performs a downmix process according to the front left channel signal, the reverberation signal, and the rear left surround sound signal to obtain a virtual stereo signal of the left channel in the dual-channel, and for example, the controller may perform a downmix process according to the front right channel signal, the reverberation signal, and the rear right surround sound signal to obtain a virtual stereo signal of the right channel in the dual-channel.
Therefore, the embodiment can generate the virtual stereo signal of which each channel is mixed with reverberation and surround sound based on the original two channel signals, restore the original spatial information lost by each channel signal, reconstruct a complex sound field, and be beneficial to improving the lost spatial sense in the recording/playback process.
In some embodiments, referring to fig. 4 and fig. 5 together, S33 includes:
s331, carrying out attenuation processing on a center channel signal to obtain a target attenuation signal;
s332, performing delay processing on the target attenuation signal to obtain a delay signal;
s333, according to a preset filter, the mixed signal of the delay signal and the center channel signal is subjected to filtering processing to obtain a reverberation signal.
In this embodiment, the target attenuation signal is a signal for simulating the energy attenuation phenomenon of sound, and as described above, when the sound wave propagates indoors, the sound wave is reflected by an obstacle such as a wall, a ceiling, or a floor, and each reflection is absorbed by the obstacle, so that the energy of the sound is attenuated, and thus this embodiment can simulate the phenomenon by the target attenuation signal. The delay signal is a signal for simulating the sound persistence phenomenon, and as described above, when the sound source stops, the sound wave is reflected and absorbed for a plurality of times in the room, and finally disappears, that is, the sound persistence phenomenon still exists after the sound source stops sounding in the room, so that the embodiment simulates the sound persistence phenomenon by performing delay processing on the target attenuation signal. The mixed signal of the delay signal and the center channel signal is subjected to filtering processing to be used for reserving a signal of a target frequency band, wherein the signal of the target frequency band can be a low frequency band signal or a high frequency band signal, and the controller can select an appropriate filter to perform filtering processing on the mixed signal of the delay signal and the center channel signal according to the target frequency band, for example, when the signal of the target frequency band is a low frequency signal, the preset filter can be a low-pass filter correspondingly, and when the signal of the target frequency band is a high frequency signal, the preset filter can be a high-pass filter correspondingly. In some embodiments, the signal in the target frequency band is a voice signal, and since the energy of the voice signal is mainly concentrated in the low frequency band, the controller may perform filtering processing on the mixed signal of the delayed signal and the center channel signal according to the low-pass filter, so as to preserve most of the energy of the voice.
In some embodiments, the controller multiplies the center channel signal by a preset attenuation factor to obtain a target attenuation signal, where the preset attenuation factor may be set according to actual requirements, and a value of the preset attenuation factor is not limited herein.
In some embodiments, as shown in fig. 5, the controller uses each frame of reverberation signal output by the preset filter as a feedback signal, and calculates the target attenuation signal according to the feedback signal, the center channel signal and the preset attenuation factor.
In some embodiments, the controller multiplies the center channel signal by a preset attenuation factor to obtain a pre-attenuation signal, and adds a feedback signal (reverberation signal) to the pre-attenuation signal to obtain a target attenuation signal.
Therefore, by the method, the reverberation signal output by the preset filter at the previous moment influences the reverberation signal output by the preset filter at the next moment, so that the required reverberation signal can be conveniently obtained.
In some embodiments, the predetermined filter is a second order low pass IIR (Infinite Impulse Response, wireless impulse response) filter.
The second order low pass IIR filter is formulated as follows:
a 0 y(n)＝b 0 x(n)+b 1 x(n-1)+b 2 x(n-2)-a 1 y(n-1)-a 2 y(n-2)
wherein y (n) is a reverberation signal of the filtering output, a 0 、a 1 、a 2 、b 1 、b 2 And, b 3 For coefficients of the second-order low-pass IIR filter, the coefficients determine the frequency curve response and gain of the second-order low-pass IIR filter, and the coefficients are formulated as follows:
b 1 ＝1-cos
a 0 ＝1+alpha
a 1 ＝-2*cos
a 2 ＝1-alpha
cons＝cos(omega)
alpha＝sin(2*Q)
wherein f 0 For the center frequency, f s For sampling frequency, Q is the quality factor.
In some embodiments, referring to fig. 6, S34 includes:
s341, performing Fourier transform processing on the time domain differential signals to obtain frequency domain differential signals;
in this step, the fourier transform process is used to transform the time domain differential signal into the frequency domain for analysis, and the controller uses the fourier transform algorithm to transform the time domain differential signal into the frequency domain to obtain the frequency domain differential signal.
To reduce spectral energy leakage, in some embodiments, the controller may first window the time domain differential signal using a window function to truncate the time domain differential signal and then fourier transform the windowed time domain differential signal to obtain a frequency domain differential signal.
In some embodiments, the window function may be any suitable type of window function, such as a hanning window, triangular window, rectangular window, and the like. Taking the hanning window as an example, the frequency domain differential signal is formulated as follows:
FD＝fft(td.win)
FD is a frame frequency domain differential signal, fft () represents fourier transform, td is a frame time domain differential signal, win represents hanning window, size of hanning window is 2×part_len, and part_len is length of a frame time domain differential signal.
In some embodiments, the fourier transform is a fast fourier transform (Fast Four ier Transform, fft), which is a fast algorithm of a discrete fourier transform (Di screte Four ier Transform, DFT) that is used to transform the signal from the time domain to the frequency domain in order to analyze the spectral structure and the law of variation.
S342, calculating a frequency domain gain signal according to the frequency domain differential signal and a preset gain factor;
in this step, the controller multiplies the frequency domain differential signal by a preset gain factor to obtain a frequency domain gain signal, where the preset gain factor can be set according to actual requirements, and the value of the preset gain factor is not limited here. The frequency domain gain signal is formulated as follows:
FG＝(FD.Fre_gain)
where FG is a frame frequency domain gain signal, FD is a frame frequency domain differential signal, and fre_gai n is a predetermined gain factor.
S343, performing inverse Fourier transform processing on the frequency domain gain signal to obtain a surround sound signal.
In this step, an inverse fourier transform process is used to convert the frequency domain gain signal to the time domain for analysis. The surround sound signal is formulated as follows:
sound＝ifft(FG)
where sound is a frame surround signal, ifft () represents the inverse fourier transform, and FG is a frame frequency domain gain signal.
It should be noted that, after the controller obtains multi-frame surround signals, the controller may connect the frame-by-frame surround signals into smooth continuous surround signals to generate virtual stereo signals subsequently, where the overlap-add principle is a fast convolution principle that extends the segment sequence and the system sequence by zero padding so that the circumferential convolution result is the same as the linear convolution result, and no confusion occurs, and the principle is proposed for calculating an infinitely long sequence linear convolution, where the circumferential convolution is required to replace the linear convolution to facilitate computer calculation.
In some embodiments, the first channel signal, the reverberation signal, and the surround sound signal have corresponding gain coefficients, and in step S25, the controller calculates gain signals corresponding to the first channel signal, the reverberation signal, and the surround sound signal according to the first channel signal, the reverberation signal, the surround sound signal, and the corresponding gain coefficients, where the gain signals corresponding to the first channel signal are the first channel gain signal, the gain signals corresponding to the reverberation signal are the reverberation gain signal, and the gain signals corresponding to the surround sound signal are the surround sound gain signal.
In some embodiments, the controller calculates the first channel gain signal from the first channel signal and a preamble gain coefficient, wherein the preamble gain coefficient is used to adjust a preamble gain of the first channel signal. Specifically, the controller multiplies the first channel signal by the preamble gain coefficient to obtain the first channel gain signal, where the first channel gain signal is the front left channel gain signal, and where the first channel signal is the front right channel gain signal, the first channel gain signal is the front right channel gain signal.
The front left channel gain signal is formulated as follows:
FL＝data_l*K1
where FL is the front left channel gain signal, data_l is the front left channel gain signal, and K1 is the preamble gain coefficient.
The front right channel gain signal is formulated as follows:
FR＝data_r*K1
where FR is the front right channel gain signal, data_l is the front right channel gain signal, and K1 is the preamble gain coefficient.
The controller calculates a reverberation gain signal according to the reverberation signal and a central gain coefficient, wherein the central gain coefficient is used for adjusting the central gain of the reverberation signal. Specifically, the controller multiplies the reverberation signal by a central gain coefficient to obtain a reverberation gain signal.
The reverberant gain signal is formulated as follows:
C＝Reverb*K2
wherein, C is the reverberation gain signal, reverb is the reverberation signal, and K2 is the central gain coefficient.
The controller calculates a surround sound gain signal according to the surround sound signal and a spatial gain coefficient, wherein the spatial gain coefficient is used for adjusting the spatial gain of the surround sound signal. Specifically, the controller multiplies the surround sound signal by a spatial gain coefficient to obtain a surround sound gain signal. As described above, when the surround sound signal is a left rear surround sound signal, the surround sound gain signal is a left rear surround sound gain signal, and when the surround sound signal is a right rear surround sound signal, the surround sound gain signal is a right rear surround sound gain signal.
The left rear surround sound gain signal is formulated as follows:
RL＝sound_l*K3
where RL is the left rear surround gain signal, sound_l is the left rear surround signal, and K3 is the spatial gain coefficient.
The right rear surround sound gain signal is formulated as follows:
RR＝sound_r*K3
then, the controller fuses the first channel gain signal, the reverberation gain signal and the surround sound gain signal to generate a virtual stereo signal. As described above, when the first channel gain signal is a front left channel gain signal, the virtual stereo signal (left channel) may be generated by fusing the front left channel gain signal, the reverberation gain signal, and the rear left surround sound gain signal, and when the first channel gain signal is a front left channel gain signal, the virtual stereo signal (right channel) may be generated by fusing the front right channel gain signal, the reverberation gain signal, and the rear right surround sound gain signal.
Specifically, the controller performs summation processing on the first channel gain signal, the reverberation gain signal and the surround gain signal to generate a virtual stereo signal. As described above, for the virtual stereo signal of the left channel, the controller may sum the front left channel gain signal, the reverberation gain signal, and the rear left surround gain signal to obtain the virtual stereo signal of the channel, and for the virtual stereo signal of the right channel, the controller may sum the front right channel gain signal, the reverberation gain signal, and the rear right surround gain signal to obtain the virtual stereo signal of the channel.
The virtual stereo signal for the left channel is formulated as follows:
Out_L＝FL+C+RL
where out_l is the virtual stereo signal of the left channel, FL is the front left channel gain signal, C is the reverberant gain signal, and RL is the front left surround gain signal.
The virtual stereo signal for the right channel is formulated as follows:
Out_R＝FR+C+RR
wherein out_r is a virtual stereo signal of a right channel, FR is a gain signal of a right front channel, C is a reverberation gain signal, and RR is a gain signal of a right front surround sound.
It should be noted that, in the foregoing embodiments, there is not necessarily a certain sequence between the steps, and those skilled in the art will understand that, according to the description of the embodiments of the present invention, the steps may be performed in different orders in different embodiments, that is, may be performed in parallel, may be performed interchangeably, or the like.
As another aspect of the embodiments of the present invention, the embodiments of the present invention provide a virtual stereo generating apparatus. Referring to fig. 7, the virtual stereo generating apparatus 700 includes an obtaining module 71, a first calculating module 72, a second calculating module 73, a first generating module 74, a second generating module 75, and a third generating module 76.
The acquisition module 71 is configured to acquire a first channel signal and a second channel signal, the first calculation module 72 is configured to calculate a center channel signal according to the first channel signal and the second channel signal, the second calculation module 73 is configured to calculate a time domain difference signal according to the first channel signal and the second channel signal, the first generation module 74 is configured to generate a reverberation signal according to the center channel signal, the second generation module 75 is configured to generate a surround sound signal according to the time domain difference signal, and the third generation module 76 is configured to generate a virtual stereo signal according to the first channel signal, the reverberation signal, and the surround sound signal.
Therefore, the embodiment can generate the virtual stereo signal of which each channel is mixed with reverberation and surround sound based on the original two channel signals, and restore the lost space information of the original channel signals so as to reconstruct a complex sound field, thereby being beneficial to improving the lost space feel in the recording/playback process.
In some embodiments, the first channel signal, the reverberation signal, and the surround sound signal have corresponding gain coefficients, referring to fig. 8, the third generating module 76 includes a first calculating unit 761 and a fusing unit 762.
The first calculating unit 761 is configured to calculate gain signals corresponding to the first channel signal, the reverberation signal, and the surround sound signal according to the first channel signal, the reverberation signal, the surround sound signal, and the corresponding gain coefficients, and the fusing unit 762 is configured to fuse the gain signals to generate a virtual stereo signal.
In some embodiments, the first computing unit 761 is specifically configured to: a first channel gain signal is calculated from the first channel signal and the first gain signal, a reverberation gain signal is calculated from the reverberation signal and the second gain signal, and a surround sound gain signal is calculated from the surround sound signal and the third gain signal.
In some embodiments, the fusing unit 762 is specifically configured to: and summing the first channel gain signal, the reverberation gain signal and the surround sound gain signal to generate a virtual stereo signal.
In some embodiments, the first computing module 72 is specifically configured to: and carrying out average processing on the first channel signal and the second channel signal to obtain a center channel signal.
In some embodiments, the second computing module 73 is specifically configured to: subtracting the second channel signal from the first channel signal to obtain a signal difference value, and dividing the signal difference value by a preset coefficient to obtain a time domain differential signal.
In some embodiments, referring to fig. 9, the first generating module 74 includes an attenuation processing unit 741, a delay processing unit 742, and a filtering processing unit 743.
The attenuation processing unit 741 is configured to attenuate the center channel signal to obtain a target attenuated signal, the delay processing unit 742 is configured to delay the target attenuated signal to obtain a delayed signal, and the filtering processing unit 743 is configured to filter a mixed signal of the delayed signal and the center channel signal according to a preset filter to obtain a reverberation signal.
The attenuation processing unit 741 specifically is configured to: and calculating a pre-attenuation signal according to the center channel signal and a preset attenuation factor, and calculating a target attenuation signal according to the pre-attenuation signal and the reverberation signal.
In some embodiments, referring to fig. 10, the second generating module 75 includes a fourier transform processing unit 751, a second computing unit 752, and an inverse fourier transform process 753.
The fourier transform processing unit 751 is configured to perform fourier transform processing on the time domain differential signal to obtain a frequency domain differential signal, the second calculating unit 752 is configured to calculate a frequency domain gain signal according to the frequency domain differential signal and a preset gain factor, and the inverse fourier transform processing unit 753 is configured to perform inverse fourier transform processing on the frequency domain gain signal to obtain a surround sound signal.
The virtual stereo generating device may execute the virtual stereo generating method provided by the embodiment of the present invention, and has the functional module and beneficial effects corresponding to the executing method. Technical details not described in detail in the embodiment of the virtual stereo generating apparatus may be referred to the virtual stereo generating method provided in the embodiment of the present invention.
Referring to fig. 11, fig. 11 is a schematic diagram of a hardware circuit structure of a controller according to an embodiment of the invention. As shown in fig. 11, the controller 17 includes one or more processors 171 and a memory 172. In fig. 11, a processor 171 is taken as an example.
The processor 171 and the memory 172 may be connected by a bus or otherwise, for example in fig. 11.
The memory 172 is a non-volatile computer readable storage medium, and may be used to store a non-volatile software program, a non-volatile computer executable program, and modules, such as program instructions/modules corresponding to the virtual stereo sound generating method in the embodiment of the present invention. The processor 171 executes various functional applications and data processing of the virtual stereo sound generating apparatus by running nonvolatile software programs, instructions and modules stored in the memory 172, that is, implements the virtual stereo sound generating method provided by the above method embodiment and the functions of the respective modules or units of the above apparatus embodiment.
The memory 172 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 172 may optionally include memory located remotely from processor 171, which may be connected to processor 171 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The program instructions/modules are stored in the memory 172, which when executed by the one or more processors 171, perform the virtual stereo generation method in any of the method embodiments described above.
Embodiments of the present invention also provide a non-transitory computer storage medium storing computer executable instructions for execution by one or more processors, such as the one processor 171 of fig. 11, to cause the one or more processors to perform the method of generating virtual stereo in any of the method embodiments described above.
Embodiments of the present invention also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by an electronic device, cause the electronic device to perform the method of generating virtual stereo according to any one of the preceding claims.
The above-described embodiments of the apparatus or device are merely illustrative, in which the unit modules illustrated as separate components may or may not be physically separate, and the components shown as unit modules may or may not be physical units, may be located in one place, or may be distributed over multiple network module units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the respective embodiments or some parts of the embodiments.
Finally, it is to be noted that the present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, which are not to be construed as additional limitations on the scope of the invention, but rather as providing for a more thorough understanding of the present invention. And under the idea of the invention, the technical features described above are continuously combined with each other, and many other variations exist in different aspects of the invention as described above, which are all considered as the scope of the description of the invention; further, modifications and variations of the present invention may be apparent to those skilled in the art in light of the foregoing teachings, and all such modifications and variations are intended to be included within the scope of this invention as defined in the appended claims.
Claims (10)
1. A method of generating virtual stereo sound, comprising:
acquiring a first sound channel signal and a second sound channel signal;
calculating a center channel signal and a time domain differential signal according to the first channel signal and the second channel signal;
generating a reverberation signal from the center channel signal;
generating a surround sound signal according to the time domain differential signal;
and generating a virtual stereo signal according to the first channel signal, the reverberation signal and the surround sound signal.
2. The method of generating of claim 1, wherein the first channel signal, the reverberant signal, and the surround sound signal have corresponding gain coefficients, the generating a virtual stereo signal from the first channel signal, the reverberant signal, and the surround sound signal comprising:
calculating gain signals corresponding to the first channel signal, the reverberation signal and the surround sound signal according to the first channel signal, the reverberation signal, the surround sound signal and the corresponding gain coefficients;
and fusing the gain signals to generate a virtual stereo signal.
3. The method of generating of claim 2, wherein calculating gain signals corresponding to the first channel signal, the reverberant signal, and the surround sound signal based on the first channel signal, the reverberant signal, the surround sound signal, and the corresponding gain coefficients comprises:
calculating a first channel gain signal according to the first channel signal and the leading gain coefficient;
calculating a reverberation gain signal according to the reverberation signal and the central gain coefficient;
and calculating a surround sound gain signal according to the surround sound signal and the space gain coefficient.
4. A method of generating according to claim 3, wherein said fusing each of said gain signals to generate a virtual stereo signal comprises:
and summing the first channel gain signal, the reverberation gain signal and the surround sound gain signal to generate a virtual stereo signal.
5. The method of generating of claim 1, wherein the calculating a center channel signal from the first channel signal and the second channel signal comprises:
and carrying out average processing on the first channel signal and the second channel signal to obtain a center channel signal.
6. The method of generating of claim 1, wherein calculating a time domain difference signal from the first channel signal and the second channel signal comprises:
subtracting the second channel signal from the first channel signal to obtain a signal difference value;
dividing the signal difference by a preset coefficient to obtain a time domain differential signal.
7. The method of generating of claim 1, wherein the generating a reverberation signal from the center channel signal comprises:
performing attenuation processing on the center channel signal to obtain a target attenuation signal;
performing delay processing on the target attenuation signal to obtain a delay signal;
and filtering the mixed signal of the delay signal and the center channel signal according to a preset filter to obtain a reverberation signal.
8. The method of generating of claim 7, wherein attenuating the center channel signal to obtain an attenuated signal comprises:
calculating a pre-attenuation signal according to the center channel signal and a preset attenuation factor;
and calculating a target attenuation signal according to the pre-attenuation signal and the reverberation signal.
9. The method of generating of any of claims 1 to 8, wherein the generating a surround sound signal from the time domain differential signal comprises:
performing Fourier transform on the time domain differential signals to obtain frequency domain differential signals;
calculating a frequency domain gain signal according to the frequency domain differential signal and a preset gain factor;
and performing inverse Fourier transform processing on the frequency domain gain signal to obtain a surround sound signal.
10. An electronic device, comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the generation method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310237775.XA CN116320908A (en) | 2023-03-01 | 2023-03-01 | Virtual stereo generation method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310237775.XA CN116320908A (en) | 2023-03-01 | 2023-03-01 | Virtual stereo generation method and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116320908A true CN116320908A (en) | 2023-06-23 |
Family
ID=86823616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310237775.XA Pending CN116320908A (en) | 2023-03-01 | 2023-03-01 | Virtual stereo generation method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116320908A (en) |
-
2023
- 2023-03-01 CN CN202310237775.XA patent/CN116320908A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11184727B2 (en) | Audio signal processing method and device | |
| JP5285626B2 (en) | Speech spatialization and environmental simulation | |
| US8126172B2 (en) | Spatial processing stereo system | |
| EP1927264B1 (en) | Method of and device for generating and processing parameters representing hrtfs | |
| CN102395098B (en) | Method of and device for generating 3D sound | |
| US7412380B1 (en) | Ambience extraction and modification for enhancement and upmix of audio signals | |
| CN107750042B (en) | generating binaural audio by using at least one feedback delay network in response to multi-channel audio | |
| US7613305B2 (en) | Method for treating an electric sound signal | |
| US9729991B2 (en) | Apparatus and method for generating an output signal employing a decomposer | |
| CN104010264B (en) | The method and apparatus of binaural audio signal process | |
| Farina et al. | Ambiophonic principles for the recording and reproduction of surround sound for music | |
| CN114401481A (en) | Generating binaural audio by using at least one feedback delay network in response to multi-channel audio | |
| CA2744429C (en) | Converter and method for converting an audio signal | |
| AU2017210021A1 (en) | Synthesis of signals for immersive audio playback | |
| US8774418B2 (en) | Multi-channel down-mixing device | |
| JP2019508978A (en) | Subband space crosstalk cancellation for audio playback | |
| Rafaely et al. | Spatial audio signal processing for binaural reproduction of recorded acoustic scenes–review and challenges | |
| CN113808569B (en) | Reverberation construction method and related equipment thereof | |
| CN105684465B (en) | Sound spatialization with interior Effect | |
| CN117376784A (en) | Method for expanding mono stereo field, electronic device, and storage medium | |
| US9794717B2 (en) | Audio signal processing apparatus and audio signal processing method | |
| Liitola | Headphone sound externalization | |
| CN116320908A (en) | Virtual stereo generation method and electronic equipment | |
| CN108810737B (en) | Signal processing method and device and virtual surround sound playing equipment | |
| KR20030002868A (en) | Method and system for implementing three-dimensional sound |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |