WO2017185663A1

WO2017185663A1 - Method and device for increasing reverberation

Info

Publication number: WO2017185663A1
Application number: PCT/CN2016/102026
Authority: WO
Inventors: 徐德著; 赵翔宇; 霍夫曼帕布洛
Original assignee: 华为技术有限公司
Priority date: 2016-04-27
Filing date: 2016-10-13
Publication date: 2017-11-02
Also published as: CN105792090B; CN105792090A

Abstract

Provided in embodiments of the present invention are a method and device for generating reverberation. A room impulse response (RIR) of an interior environment is acquired, the RIR expressing time-domain information of reverberation of the interior; target head-related impulse response (HRIR) data in an audio system is determined, and the energy and length of the target HRIR data are calculated; the RIR is normalized on the basis of the energy of the target HRIR data to produce a first signal; the first signal is faded in on the basis of the length of the target HRIR data to produce a second signal; distance information is determined, the distance information being used for expressing the distance at which a virtual audio source is distanced from a user; and the energy of the second signal is adjusted on the basis of the distance information to produce a late reverberation signal. Facilitated is the production of true sampling reverberation in a timely manner; at the same time, the sampling reverberation produced is processed, gain control that is energy normalized and combined with distance information is added, extracephalic sensations in an audio system can be intensified, and different perceptions of distance can be produced with the distance information.

Description

Method and device for increasing reverberation

Technical field

The present invention relates to the field of communications technologies, and in particular, to a method and apparatus for increasing reverberation.

Background technique

The current research in the field of virtual reality is very extensive, and the experience of virtual audio is getting more and more attention. At present, the VR audio effect mainly uses the head related transfer function HRTF (Head Related Transfer Function) technology combined with the head tracking to realize the sound source does not follow the head effect, and simulates the real environment audio experience.

For a time domain signal-head related impulse response HRIR (Head Related Impulse Response), the frequency domain signal can be obtained by the fast Fourier transform FFT, that is, the HRTF signal, and the HRTF signal can be obtained by the IFFT signal. Can be regarded as an equivalence relationship. The VR audio system generally includes an HRIR or HRTF database, such as a well-known database such as CIPIC (Center for Image Processing and Integrated Computing), which includes HRTF/HRIR data corresponding to different orientations on the three-dimensional spherical surface.

The sound wave emitted by the sound source is diffracted by the head, the auricle, the trunk, etc., and the physical process can be regarded as a linear time-invariant acoustic filtering system. Its characteristics can be described by the frequency domain transfer function of the system, which is defined as Head related transfer function. It reflects the filtering effect of the outer ear, head and torso on the sound signal when the orientation of the sound signal entering the human ear is different. Therefore the HRTF data contains direction information.

A common audio processing technique is shown in FIG. 1. The HRIR database is converted into an HRTF database in advance, and the corresponding HRTF data is selected according to the orientation information obtained by the head-tracking processing. According to the principle that the time domain convolution is equivalent to the frequency domain product, the HRTF data is multiplied by the FFT transformation result of the input time domain audio signal such as the video track, and then the IFFT is converted into the time domain signal and the time domain signal is used in the earphone. Play to the user.

However, this kind of processing technology only realizes the sound with a sense of direction, and the effects of the head effect and the sound source distance sense inherent in the HRTF technology are still defective. The so-called head-in effect is specifically when listening to music. As if vocals and instruments exist in the brain rather than in space, this effect should be minimized or avoided. In addition to the head effect and the head external effect, the head external effect means that when the user listens to music with headphones, the listener seems to be in the center of a concert hall, and he is surrounded by the sounds of various instruments around him. This effect should be Enhanced. Correspondingly, the head effect and the head external effect are the same, such as increasing the head external effect and reducing the head effect. On this basis, users also need to be able to feel the proximity of different sound sources, and this effect should also be enhanced.

When sound waves are transmitted indoors, they must be reflected by obstacles such as walls, ceilings, and floors. Each reflection must be absorbed by obstacles. In this way, when the sound source stops sounding, the sound wave is reflected and absorbed multiple times in the room, and finally disappears. We feel that the sound continues for a while after the sound source stops sounding. This phenomenon is called reverberation. Reverb contains spatial information in the room, so adding reverberation can enhance the head external effect and weaken the head effect.

For the measurement of an indoor reverberation, a short acoustic test signal is generally transmitted through the speaker indoors. After a series of reflections, the sound is picked up by the microphone, and the indoor space can be obtained by specific processing for different types of test signals. The time domain signal of the reverb.

The Dolby processing is a process of adding late reverberation based on the technique shown in FIG. 1. Dolby uses the well-known FDN (Feedback Delay Network) method to add post-reverberation solution to the head effect. In some instances, HRTF convolution processing is performed on each channel of the multi-channel input, while the next reverberation is performed. The channel does at least one FDN processing. A specific example is shown in FIG. 2: the input signal is a multi-channel time domain signal such as a video track; and the signal of each channel in the input signal is convoluted by the direct response pre-reflection processing module 200 and the HRTF to obtain a corresponding The time domain two-channel output signal directly adds the outputs of the different channels to obtain the first two-channel output signal; adds the different channel signals in the input signal, and reverberates through the module under the reverberation module 201 Into a mono signal, the mono signal is processed by the FDN module (220) to obtain a second two-channel output signal; the first two-channel output signal and the second two-channel output signal are mixed The ringing module 210 adds reverberation to obtain the final output signal, and feeds back to the user with the earphone.

For the FDN module (220), at least one FDN module operation is required, and each FDN module needs all-pass filtering, multiple delay processing, and each audio time domain sample needs to perform a matrix operation (see patent WO 2015/102920 A1). Therefore, the calculation modules and parameter settings involved are very In addition, FDN is artificial reverberation, different scenes need to set different FDN parameters, so it can not be flexibly applied to a variety of application scenarios; the effect is difficult to be consistent with the real environment reverberation; and the control function of distance sense is not realized.

Summary of the invention

In view of this, embodiments of the present invention provide a method and apparatus for increasing reverberation.

In a first aspect, an embodiment of the present invention provides a method for generating reverberation: acquiring a room impulse response RIR of an indoor environment, the RIR characterizing time domain information of the reverberation in the indoor; and determining a target head related impulse response HRIR in the audio system Data, and calculating energy and length of the target HRIR data; normalizing the RIR according to the energy of the target HRIR data to obtain a first signal; and performing fade-in processing on the first signal according to the length of the target HRIR data Two signals; determining distance information, wherein the distance information is used to represent a distance of the virtual sound source from the user; and the second signal is energy-adjusted according to the distance information to generate a reverberation signal. In general, the device can easily acquire the RIR through its own speaker and microphone, eliminating the need for external equipment; processing the resulting sampled reverb, adding energy normalization and gain control combined with distance information, can enhance the head in the audio system External sense, and different distance senses are generated by distance information.

In a second aspect, an embodiment of the present invention provides a method for adding reverberation: the foregoing method is applied to a VR device, where the VR device includes a head sensor, where the head sensor is used to obtain head orientation sensing information; : acquiring a room impulse response RIR of the indoor environment, the RIR characterizing the time domain information of the reverberation in the indoor; determining the target head related impulse response HRIR data in the audio system, and calculating the energy and length of the target HRIR data; according to the target HRIR The energy of the data is normalized to obtain the first signal; the first signal is fade-in processed according to the length of the target HRIR data to obtain a second signal; the distance information is determined, and the distance information is used to represent the virtual sound source distance. use a distance of the household; performing energy adjustment on the second signal according to the distance information to generate a reverberation signal; obtaining target head orientation sensing information, and determining a head related transfer function HRTF corresponding to the target head orientation sensing information; The HRTF processes the audio input signal to obtain a first audio signal; and processes the audio input signal according to the reverberation signal to obtain a second audio signal; and performs the first audio signal and the second audio signal The reverb produces an output audio signal. Generally, the VR device can easily obtain the RIR through the speaker and microphone provided by the smart terminal or the smart terminal, and no external device is needed; the obtained sample reverberation is processed, and the energy normalization and the gain control of the combined distance information are added. The head external sense in the VR audio system can be enhanced, and different distance senses are generated by the distance information.

According to the first aspect or the second aspect, in a possible design, the obtaining the room impulse response RIR of the current indoor environment comprises: transmitting an acoustic reference test signal, collecting the reflected signal of the acoustic reference test signal; and performing the reflected signal Sampling to obtain the above RIR. The emission test signal may be a preset empirical reference test signal. This makes it easier to get reverb information from the room.

According to the first aspect or the second aspect, in a possible design, the determining the target head related impulse response HRIR data in the audio system comprises: selecting the largest absolute value of the sample points in all HRIR data waveforms in the audio system HRIR database. The HRIR data is used as the target head related impulse response HRIR data. Because the HRIR is for energy normalization, selecting the largest absolute value of the sampling point normalizes the RIR to an energy level that best matches the HRIR database.

In a third aspect, an embodiment of the present invention provides an apparatus for generating reverberation, the apparatus comprising: an acquisition module, configured to acquire a room impulse response RIR of an indoor environment, where the RIR characterizes time domain information of the reverberation in the indoor; a determining module, configured to determine a target head related impulse response HRIR data in the audio system, and calculate energy and length of the target HRIR data; a first calculating module, configured to The energy of the target HRIR data calculated by the module is normalized to obtain the first signal by the RIR obtained by the obtaining module, and the second calculating module is configured to calculate the length of the target HRIR data calculated by the first determining module. The first signal is subjected to the fade-in processing to obtain the second signal; the second determining module is configured to determine the distance information, where the distance information is used to represent the distance of the virtual sound source from the user; and the third calculating module is configured to determine according to the second determining module. The distance information is energy-adjusted to the second signal calculated by the second calculation module to generate a reverberation signal. In general, the device can easily acquire the RIR through its own speaker and microphone, eliminating the need for external equipment; processing the resulting sampled reverb, adding energy normalization and gain control combined with distance information, can enhance the head in the audio system External sense, and different distance senses are generated by distance information.

In a fourth aspect, an embodiment of the present invention provides a device for generating reverberation, the device being applied to a VR device, where the VR device includes a head sensor, where the head sensor is used to obtain head orientation sensing information; And an obtaining module, configured to obtain a room impulse response RIR of the indoor environment, wherein the RIR characterizes time domain information of the reverberation in the indoor; the first determining module is configured to determine a target head related impulse response HRIR data in the audio system, and calculate the above a first calculation module, configured to normalize the RIR obtained by the obtaining module according to the energy of the target HRIR data calculated by the first determining module to obtain a first signal; and the second computing module And performing a fade-in process on the first signal according to the length of the target HRIR data calculated by the first determining module to obtain a second signal; and a second determining module, configured to determine distance information, where the distance information is used to represent the virtual sound source a distance from the user; a third calculation module, configured to determine the distance information according to the second determining module The second signal calculated by the second calculation module performs energy adjustment to generate a reverberation signal; and a fourth calculation module is configured to process the audio input signal according to the head related transfer function HRTF to obtain a first audio signal, where The HRTF is determined by the head orientation sensing information; and the fifth calculation module is configured to calculate according to the third The reverberation signal calculated by the calculation module processes the audio input signal to obtain a second audio signal, and the sixth calculation module is configured to calculate the first audio signal and the fifth calculation module calculated by the fourth calculation module. The calculated second audio signal is reverberated to obtain an output audio signal.

According to the third aspect or the fourth aspect, in a possible design, the acquiring module is specifically configured to: transmit an acoustic reference test signal, collect a reflected signal of the acoustic reference test signal; and sample the reflected signal to obtain the RIR. . The emission test signal may be a preset empirical reference test signal. This makes it easier to get reverb information from the room.

According to the third aspect or the fourth aspect, in a possible design, the first determining module is specifically configured to: select, as the target, the maximum HRIR data of the absolute values of the samples in all HRIR data waveforms in the audio system HRIR database. The head related impulse responds to HRIR data.

In a fifth aspect, an embodiment of the present invention provides an apparatus for enhancing reverberation, where the apparatus includes: a processor, a memory, a transceiver, and a bus system; wherein the processor and the memory and the transceiver are connected by using the bus system, The above memory is used for storing programs and instructions, the transceiver is for transmitting and receiving audio signals, and the processor executes any of the above possible implementation methods by calling the programs and instructions in the above memory.

According to the technical solution provided by the embodiment of the present invention, the RIR of the current room can be easily obtained by using the speaker and the microphone of the device without using other audio collection and testing equipment, and the mathematical method of the RIR is processed to obtain an audio system. A reverberation signal that can be used to convolute the input signal of the audio so that the output signal can increase the head external effect of the user, mitigate the effect in the head, and create artificial reverberation without additionally setting complicated parameter information. The invention can bring a better audio experience to the user.

DRAWINGS

1 is a common audio processing technique in the prior art;

2 is a schematic diagram of signal processing of a late reverberation;

3 is a schematic diagram of an application device according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an application scenario combined with specific hardware according to an embodiment of the present invention; FIG.

FIG. 5 is a schematic diagram of an audio processing process for adding reverberation according to an embodiment of the present invention; FIG.

6 is a flowchart of a method for generating a reverberation signal according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of an apparatus for generating reverberation according to an embodiment of the present invention; FIG.

FIG. 8 is a schematic diagram of an apparatus for increasing reverberation according to an embodiment of the present invention; FIG.

FIG. 9 is a schematic structural diagram of an apparatus for generating reverberation according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention provide a method and apparatus for increasing reverberation.

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of preferred embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The application scenario of the present invention is a smart wearable device, a smart audio and video system, and the like, such as a headband VR device, which includes a smart terminal device, such as a smart terminal such as a mobile phone or a tablet (hereinafter, a mobile phone is used as an example, if the VR device includes a function) Other terminals equivalent to the mobile phone should also belong to the protection scope of the present invention. When the user is in different indoor spaces, the reverberation command can be issued through the mobile phone to obtain the reverberation information of the current indoor space, and according to the current indoor The reverberation information is post-processed to the sound in the VR device, and the processed sound is returned to the user through the earphone.

Referring to FIG. 3, in a specific implementation process, the device to which the present invention is applied is a smart terminal combined with the VR device 300, such as a smart phone 304. The smart phone includes a speaker 306 and a microphone 305, and the mobile phone is inserted into the VR device. As a VR display device, as well as an audio signal processing device, a pair of connected VR devices The earphone device 302 is configured to play a virtual audio signal to the user. The smart phone also includes a storage unit 307 and a computing unit 308. The control unit 301 in the VR device accepts a user command. When the user triggers the update of the reverberation command, the smart phone 304 reverbers to the real environment room through the microphone 305 and the speaker 306. The measurement is performed and processed by the calculation unit 308 to obtain a late reverberation signal, which is stored in the storage unit 307 together with the HRTF database. When the user experiences VR audio, computing unit 308 is responsible for the operation of the VR audio system, and the output time domain audio signal is ultimately played back to the user with headset device 302.

The VR device 300 and the smart phone 304 can be connected by wire. Generally, the VR device 300 and the smart phone 304 are integrated in this case; the VR device 300 and the smart phone 304 can also pass Bluetooth, Wifi, The wireless communication method such as infrared performs wireless connection communication. Usually, the VR device 300 and the smart phone 304 are separated in this case.

The present invention utilizes a speaker and a microphone on a smart phone that is used with a VR device, and does not require additional test equipment. It is possible to quickly measure and update the current real environment reverberation information without having to make a lot of reverberation parameter settings. After obtaining the real room reverberation information, a series of processes in the present invention are performed to obtain a usable late reverberation signal, and a usable late reverberation signal is obtained, and further gain control is combined with distance information for use in VR audio processing.

A schematic diagram of an application scenario combining specific hardware is shown in FIG. 4: 10 is a smart phone with a signal control and processing unit, 11 is a microphone on a smart phone, 12 is a speaker on a smart phone, and 13 is a head mounted VR device.

A more visual audio processing flow can be seen in Figure 5. The input signal is a multi-channel time domain signal such as a video track; the processing flow is as follows:

1) determining head orientation sensing information according to the head sensor in the VR device, and matching the HRTF function corresponding to the head orientation sensing information in the HRTF database according to the head orientation sensing information; One channel of the signal is convoluted by the HRTF binaural audio processing module 500 and the matched HRTF function to obtain a corresponding time domain two-channel output signal S1, and the outputs of the different channels are directly added to obtain an output signal S1;

2) obtaining the late reverberation signal S2 through the late reverberation processing module 503;

3) Adding different channel signals in the input signal, and reverberating into one by the module under reverberation module 501 a mono signal, the mono signal and the late reverberation signal S2 are FFT fast convolved by the FFT fast convolution module 502 to obtain an output signal S3;

4) The output signal S1 and the output signal S3 are summed and reverberated by the reverberation module 504 to obtain a final output signal S4, which is fed back to the user with the earphone.

Among the above steps, step 2) is a key step of the present invention, and how to obtain the late reverberation signal S2 will be described below in a more detailed embodiment. See Figure 6 for the method steps for obtaining the late reverberation signal S2. Specific steps are as follows:

In step 610, the VR device is used to transmit an acoustic reference test signal, such as a test signal S610 such as TSP/MLS, with the speaker of the mobile phone; and the test signal is used to measure the room reverberation.

The TSP signal refers to a well-known signal that responds to the measurement signal, and can be stored in a storage device in a format such as wav, and is played by a smartphone speaker, and the sound wave is reflected by various obstacles in the room, thereby reflecting the reverberation information of the room.

In addition, the test signal is not limited to the TSP signal, and may be a maximum length sequence (MLS) or a sine sweep signal; the issuance of the test signal is controlled by the user.

In step 620, the VR device is used to match the microphone of the mobile phone or the microphone of the VR device to pick up the acoustic feedback signal corresponding to the microphone pickup test signal S610 provided by the acoustic echo signal device, and the room impulse response S620 is calculated by the calculation unit.

The above acoustic feedback signal includes signals that the sound waves are reflected by different obstacles and different times. The RIR room impulse response is calculated by the calculation unit, which represents the time domain waveform of the room reverberation information. The RIR of different rooms have different room impulse responses, which can be recorded as R(n), where n is [0, N], N The length of the sample of the finally selected RIR, such as N = 4096; this calculation method is a well-known method. For example, a two-dimensional waveform represents the RIR signal, the abscissa is the sample point, and the ordinate is the sample point value. The sample points represent the sampling points of the audio signal. Taking 48000 Hz sampling as an example, the interval between samples is 1/48000 seconds, that is, there are 48000 samples in 1 second.

Optionally, in this step, it is not limited to one microphone recording. If there are more than two available microphones, the two microphones whose materials are selected and are far away are preferentially selected. The two microphone recording data is processed. When there is only one microphone, only one available post-reverberation signal can be obtained. In this case, the VR audio processing can be applied to the left and right channels; when there are two microphones, there are two available post-reverberation signals. No., respectively, they are applied to the left and right channels, because the signals collected by the two microphones are not completely the same, the correlation of the reverberation is reduced, and the overall stereoscopic effect will be enhanced, but this will require two steps of the S630-S650 calculation in this scheme. . For three microphones, it is not necessary to perform a three-way operation to select two microphone signals with the closest performance and the farthest distance from the microphone.

Step 630: Perform energy normalization processing on the room impulse response S620 by the calculating unit according to the HRIR data energy used by the VR audio system to obtain a signal S630.

First, the target HRIR data of the maximum value of the absolute values of the samples in all HRIR data waveforms in the HRIR database (equivalent to the HRTF database) is statistically obtained, and the sample with the largest absolute value of the sample points in the target HRIR data. Position (time point) N1.

Secondly, the sampled position N2 corresponding to the maximum value of the absolute value of the sampled impulse response of the RIR room is statistically recorded, and the recording response peak point is aligned according to the peak sample of the HRIR data. At this time, the front (N2-N1) samples of the RIR data need to be deleted to ensure time synchronization. The aligned RIR is:

R ₁ (n)=R(n+N2-N1) n∈[1,N]

For the HRIR after alignment processing and the RIR after alignment processing, samples of the same length are intercepted, and the length of the intercept depends on the length used in the HRIR database of the VR audio system. Taking the length 512 as an example, the HRIR samples [N1, N1+511] are intercepted, and the RIR samples [N2', N2'+511] are intercepted, that is, 512 samples are respectively intercepted, and the HRIR and the RIR intercepted portions are respectively calculated. RMS rms energy E1, E2. The energy normalization is performed by multiplying the aligned RIR by E1/E2. among them,

R ₂ (n)=(E1/E2)*R ₁ (n) n∈[1,N]

R ₂ (n) is the signal S630.

Step 640, according to the HRIR data length adopted by the VR audio system, the calculation unit performs a fade-in process on the signal in the signal S630 to obtain a first late reverberation signal S640.

The purpose of this processing is to fade in, delete the direct sound and the pre-reflection part of the measurement reverb, leaving only the first late reverberation signal. Take the HRIR length as M samples (M can generally be 256, 512, etc.). The fade in window function is:

LR(n)=W(n)*R ₂ (n)

LR(n) is the signal S640.

Step 650: Perform energy adjustment on the first late reverberation signal S640 signal by the calculating unit according to the distance information set by the distance sensor or the user, to obtain a processed second, late reverberation signal S650 available to the system, where the signal S650 is The above signal S2 is used for subsequent arithmetic processing by the VR audio system.

Specifically, for the user to set the distance r (r ≥ 1), the unit is (m), the distance of the virtual sound source can be set for the user, or the sound source distance information obtained by the distance sensor in the VR device. The late reverberation signal gain is

This gain is directly loaded on the late reverb signal.

LR ₁ (n) is the signal S650.

In the above implementation process, in the steps 610-650, the algorithm of each step has several forms of deformation, and those skilled in the art should implement the signal processing method based on the prior art and the equivalent algorithm without any creative improvement. It is within the scope of protection of the present invention. It is not enumerated in the present invention.

In the embodiment of the present invention, a method for adding reverberation is provided. According to the technical solution provided by the embodiment of the present invention, the RIR of the current room can be easily obtained by using the speaker and the microphone of the device without using other audio collection and testing equipment. And the RIR is processed by a certain mathematical method to obtain a reverberation signal usable by the audio system, and the reverberation signal can perform convolution operation on the input signal of the audio, so that the output signal can increase the head external effect of the user and reduce the head. Effect, no need to set complex parameter information to create artificial reverberation.

Please refer to FIG. 7. FIG. 7 is a diagram of a device for generating reverberation according to an embodiment of the present invention, which may be an intelligent device. The terminal can also be a wearable device such as a VR device, or an audio processing module integrated in the product. The device 700 includes:

The obtaining module 701 is configured to obtain a room impulse response RIR of the indoor environment, and the RIR represents time domain information of the reverberation in the current indoor. The acquisition module can be implemented by a common wireless transceiver module, and the corresponding wireless transceiver mode can receive and transmit audio signals, which can be controlled by a specific application.

The first determining module 702 is configured to determine target head related impulse response HRIR data in the audio system, and calculate energy and length of the target HRIR data. The HRIR data can be stored by a memory, and the first determining module 702 can be implemented in a processor.

The first calculating module 703 is configured to perform normalization processing on the RIR acquired by the obtaining module 701 according to the energy of the target HRIR data determined by the first determining module 702 to obtain a first signal. The first computing module can be implemented in a processor.

The second calculating module 704 is configured to perform a fade-in process on the first signal according to the length of the target HRIR data calculated by the first calculating module 703 to obtain a second signal. The second computing module can be implemented in the processor.

The second determining module 705 is configured to determine distance information, where the distance information is used to represent a distance of the virtual sound source from the user; and the second determining module 705 can be implemented by a processor.

The third calculating module 706 is configured to perform energy adjustment on the second signal calculated by the second calculating module 704 according to the distance information determined by the second determining module 705 to generate a reverberation signal. The third calculation module 706 can be implemented by a processor.

Please refer to FIG. 8. FIG. 8 is a device for adding reverberation according to an embodiment of the present invention. The device may be an intelligent terminal, a wearable device such as a VR device, or a set of audio systems. The device 800 includes: an obtaining module 801, configured to acquire a room impulse response RIR of the indoor environment, and the RIR characterizes time domain information of the reverberation in the room; the acquiring module can be implemented by a common wireless transceiver module, and the corresponding wireless transceiver The mode can receive and transmit audio signals, which can be controlled by a specific application.

The first determining module 802 is configured to determine target head related impulse response HRIR data in the audio system, and calculate energy and length of the target HRIR data; the HRIR data may be stored by the memory.

The first calculating module 803 is configured to perform normalization processing on the RIR acquired by the obtaining module 801 according to the energy of the target HRIR data calculated by the first determining module 802 to obtain a first signal;

The second calculating module 804 is configured to perform a fade-in process on the first signal according to the length of the target HRIR data calculated by the first determining module 802 to obtain a second signal;

a second determining module 805, configured to determine distance information, where the distance information is used to represent a distance of the virtual sound source from the user;

The third calculating module 806 is configured to perform energy adjustment on the second signal calculated by the second calculating module 804 according to the distance information determined by the second determining module 805, to generate a reverberation signal;

The fourth calculating module 807 is configured to process the audio input signal according to the head related transfer function HRTF to obtain a first audio signal, where the HRTF is determined by the head orientation sensing information;

a fifth calculating module 808, configured to process the audio input signal according to the reverberation signal calculated by the third calculating module 806, to obtain a second audio signal;

The sixth calculation module 809 is configured to perform reverberation on the first audio signal calculated by the fourth calculation module 807 and the second audio signal calculated by the fifth calculation module 808 to obtain an output audio signal.

The above modules 802-809 can be implemented by the processor and complete the corresponding functions.

In a specific implementation process, the acquiring module 701/801 is specifically configured to: transmit an acoustic reference test signal, and collect a reflected signal of the acoustic reference test signal; sample the reflected signal to obtain an RIR. The first determining module 702/802 is specifically configured to: select, in the audio system HRIR database, the largest HRIR data of the absolute values of the sample points in all HRIR data waveforms as the target head related impulse response HRIR data.

The technical solution of the present invention provides a device for increasing reverberation, the device comprising the above modules 701-706, Or the above modules 801-808; the device is used to implement the implementation of the above steps 601-605 and an equivalent alternative implementation. The device can easily access the RIR through its own speaker and microphone, eliminating the need for external equipment; processing the resulting sampled reverb, adding energy normalization and gain control combined with distance information, can enhance the head external sense in the audio system And generate a different sense of distance through the distance information.

Please refer to FIG. 9. FIG. 9 is a schematic structural diagram of an apparatus for generating reverberation according to an embodiment of the present invention. The device 900 includes:

The processor 901 is configured to generate a corresponding operation control signal, send the corresponding component to the processing processing device, read and process the data in the software, and in particular, read and process the data and the program in the memory 902, so that each of the components The function module performs the corresponding function, thereby controlling the corresponding component to act according to the requirements of the instruction.

The memory 902 is used to store programs and various data, and mainly stores software units such as operating systems, applications, and function instructions, or a subset thereof, or an extended set thereof. A non-volatile random access memory may also be included, providing hardware, software, and data resources in the management computing processing device to the processor 901, supporting control software and applications.

The transceiver 903 is configured to collect, acquire or transmit information, especially an audio signal.

Each of the above hardware units can communicate via a bus connection.

In this way, the processor 901 controls the transceiver 903 to acquire the room impulse response RIR of the indoor environment by calling a program or instruction stored in the memory 902, and the processor 301 executes steps 610-650 in the above embodiment to generate an RIR signal. Reverb signal available in the audio system.

The implementation of the device 900 may be a smart terminal, or may be an intelligent audio system such as a VR device or a smart headset, a smart audiovisual system, a wearable home theater, and the like.

It will be apparent to those skilled in the art that all or part of the above steps may be performed by hardware associated with program instructions, which may be stored in a computer readable storage medium. Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented in hardware, firmware implementation, or a combination thereof.

The above embodiments are merely preferred embodiments of the technical solutions of the present invention, and are not intended to limit the present invention. protected range. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

A method of generating reverberation, characterized in that the method comprises:

Acquiring a room impulse response RIR of the indoor environment, the RIR characterizing time domain information of the reverberation in the room;

Determining a target head related impulse response HRIR data in the audio system and calculating an energy and length of the target HRIR data;

Normalizing the RIR according to the energy of the target HRIR data to obtain a first signal;

Performing a fade-in process on the first signal according to a length of the target HRIR data to obtain a second signal;

Determining distance information, the distance information being used to represent a distance of the virtual sound source from the user;

And performing energy adjustment on the second signal according to the distance information to generate a reverberation signal.
The method of claim 1 wherein said obtaining a room impulse response RIR of the current indoor environment comprises:

Transmitting an acoustic reference test signal, collecting a reflected signal of the acoustic reference test signal;

The reflected signal is sampled to obtain the RIR.
The method of claim 2 wherein said determining a target header related impulse response HRIR data in the audio system comprises:

The largest HRIR data of the absolute values of the samples in all HRIR data waveforms is selected in the audio system HRIR database as the target head related impulse response HRIR data.
A method for increasing reverberation, characterized in that the method is applied to a VR device, the VR device comprising a head sensor for obtaining head orientation sensing information; the method comprising: acquiring a room impulse response RIR of the indoor environment, the RIR characterizing time domain information of the reverberation in the room; determining target head related impulse response HRIR data in the audio system, and calculating energy and length of the target HRIR data;

Normalizing the RIR according to the energy of the target HRIR data to obtain a first signal;

Performing a fade-in process on the first signal according to a length of the target HRIR data to obtain a second signal;

Determining distance information, the distance information being used to represent a distance of the virtual sound source from the user;

Performing energy adjustment on the second signal according to the distance information to generate a reverberation signal;

Obtaining target head orientation sensing information, determining a head related transfer function HRTF corresponding to the target head orientation sensing information;

Processing the audio input signal according to the HRTF to obtain a first audio signal;

Processing the audio input signal according to the reverberation signal to obtain a second audio signal;

The first audio signal and the second audio signal are reverberated to obtain an output audio signal.
The method of claim 4 wherein said obtaining a room impulse response RIR of the current indoor environment comprises:

Transmitting an acoustic reference test signal, collecting a reflected signal of the acoustic reference test signal;

The reflected signal is sampled to obtain the RIR.
The method of claim 4 wherein said determining a target header related impulse response HRIR data in the audio system comprises:

The largest HRIR data of the absolute values of the samples in all HRIR data waveforms is selected in the audio system HRIR database as the target head related impulse response HRIR data.
A device for generating reverberation, characterized in that the device comprises:

An acquisition module, configured to acquire a room impulse response RIR of the indoor environment, the RIR characterizing time domain information of the reverberation in the room;

a first determining module, configured to determine target head related impulse response HRIR data in the audio system, and calculate energy and length of the target HRIR data;

a first calculating module, configured to perform normalization processing on the RIR acquired by the acquiring module according to the energy of the target HRIR data calculated by the first determining module to obtain a first signal;

a second calculating module, configured to perform a fade-in process on the first signal according to a length of the target HRIR data calculated by the first determining module to obtain a second signal;

a second determining module, configured to determine distance information, where the distance information is used to represent a distance of the virtual sound source from the user;

And a third calculating module, configured to perform energy adjustment on the second signal calculated by the second calculating module according to the distance information determined by the second determining module, to generate a reverberation signal.
The device according to claim 7, wherein the obtaining module is specifically configured to:

Transmitting an acoustic reference test signal, collecting a reflected signal of the acoustic reference test signal;

The reflected signal is sampled to obtain the RIR.
The device according to claim 7, wherein the first determining module is specifically configured to:

The largest HRIR data of the absolute values of the samples in all HRIR data waveforms is selected in the audio system HRIR database as the target head related impulse response HRIR data.
An apparatus for increasing reverberation, wherein the apparatus is applied to a VR device, the VR device includes a head sensor for obtaining head orientation sensing information; and the apparatus includes: acquiring a module for acquiring a room impulse response RIR of an indoor environment, the RIR characterizing time domain information of the reverberation in the room;

a first determining module, configured to determine target head related impulse response HRIR data in the audio system, and calculate energy and length of the target HRIR data;

a first calculating module, configured to perform normalization processing on the RIR acquired by the acquiring module according to the energy of the target HRIR data calculated by the first determining module to obtain a first signal;

a second calculating module, configured to perform a fade-in process on the first signal according to a length of the target HRIR data calculated by the first determining module to obtain a second signal;

a second determining module, configured to determine distance information, where the distance information is used to represent a distance of the virtual sound source from the user;

a third calculation module, configured to perform energy adjustment on the second signal calculated by the second calculation module according to the distance information determined by the second determining module, to generate a reverberation signal;

a fourth calculating module, configured to process the audio input signal according to the head related transfer function HRTF to obtain a first audio signal, wherein the HRTF is determined by the head orientation sensing information;

a fifth calculating module, configured to process the audio input signal according to the reverberation signal calculated by the third calculating module, to obtain a second audio signal;

And a sixth calculating module, configured to perform reverberation on the first audio signal calculated by the fourth computing module and the second audio signal calculated by the fifth computing module to obtain an output audio signal.
The device of claim 10, wherein the obtaining module is specifically configured to:

Transmitting an acoustic reference test signal, collecting a reflected signal of the acoustic reference test signal;

The reflected signal is sampled to obtain the RIR.
The device of claim 10, wherein the first determining module is specifically configured to:

The largest HRIR data of the absolute values of the samples in all HRIR data waveforms is selected in the audio system HRIR database as the target head related impulse response HRIR data.
An apparatus for increasing reverberation, the apparatus comprising: a processor, a memory, a transceiver, and a bus system; wherein the processor and the memory, the transceiver are connected by the bus system, The memory is for storing programs and instructions, the transceiver is for transceiving audio signals, and the processor performs the method of any of claims 1-6 by calling programs and instructions in the memory.